September 2019 – January 2020

Computer vision enabled in-store customer tracking for retail and other related purposes.

Research project / Big Data, Operations tool

Introduction

The store layout is one of the key facets of modern retailing, because it is a vital feature that influences consumer behaviour. This research examines the interaction between user behaviour and retail layout optimization by the use of close-circuit video or CCTV obtained from online data sources, utilizing data visualization.

Consumer behaviour with store layout insightful data could be information in the form of consumer position, crowd density, path pattern and uses of store overall layout by the customer, this could help retailer to gain knowledge that could play in their decision making. However not only retail that can benefits from this, any use case that have movement of people in its focus can benefits too, such as parks and public transit.



The Research

I started by taking a field study to my target of research, which is public places, and after some deliberation with my academic supervisor, it was decided retail would be the object. I conducted several study on local retail store to see how they operates, layouting and such. I then conducted interview with the store operator and dig in previous research for literature review, from that several questions were formulated.  

  • How retailer could gain insightful data regarding consumer position, density and pattern?
  • What actionable knowledge that could be useful from the insight gained by the proposed method?

These when answered would bring about beneficial insight, thus, several objective were created in order to guide the project.

  • Identify insight potential retail firms could get and possible advantage retail company could be receiving when applying the new method proposed.
  • The solution this research bring would be able to be used by the retailer to expand their knowledge database and gain insights value adding.

With the creation of this research, its hoped to be able to be create a tool of use for business owners that able to solve the problem and question that they had, especially retail to gain insight can help businesses form actionable knowledge to assist in their decision making and planning.

Research Reasoning

A vast data of each customer visit is available for observation and waiting to be analysed, however, it is a laborious task for a human to do, as they need to pinpoint each movement of the customer and at one time, there might be multiple individuals in the store to be tracked and quantified. However, the rise of technology has enabled these data to be digitized and processed by various techniques such as data mining, and it can be used to discover knowledge discovery when used as business intelligence tool.

Literature Review Summary

It is well known from a business strategy perspective that companies are always looking for ways to reduce costs and increase sales in order to optimize the gross margin contribution and retail company are no exception. The customer behaviour during a store visit thus is an abject opportunity that could be harnessed if the information is able to be gained and processed to be actionable knowledge.

One of crucial aspect in brick and mortar conventional retailing is store layout and thus can be one of the most important aspect affecting consumer behaviour. Consumer behaviour with store layout insightful data could be information in the form of consumer position, crowd density, path pattern and uses of store overall layout by the customer, this could help retailer to gain knowledge that could play in their decision making. Layout is an integral factor that affects consumer behaviour and is a crucial determining factor of store perception.

Empirical evidence supports the idea that shop layout information affects the expectations of customers regarding services and allows consumers to classify service companies. This study thus aims to provide or propose a method that can be used to help analyse consumer behaviour in relation to store layout.

The Method

The research object are individual movements and placement or location in the establishment gathered from media data in form of video. From here, we able to extrapolate and create a map or blueprint of the target premises for analysis

The blueprint layout ot the target store, with dashed line as camera view

The video are taken on elevated view platform and are produced by closed circuit television or CCTV. This enables establishment (stores, for example in this research) to catalogues and make use of existing data that have been sourced daily and use it for more useful purposes. The premise layout that captured by the CCTV or another form of media capture would then be processed by the model.

Model Processing

The data that would be used are the video taken of the establishment, for the figure presented the establishment is a store entrance viewpoint. The model would then take every frame data of the video to be processed by the model algorithm, it works by assigning IDs to each unique object detected and recording it within its memory and outputting it as series of coordinates ID. This process are repeated until all frames of the video went through the algorithm and each object have been catalogued and recorded into coordinates.



There are four steps. First, the process would take in the data from CCTV cameras, The second step would take each snapshot and feed it into the neural network and each object in the video to be tracked and the position to be recorded, in this steps several parameters to the system would be conducted to see which is the best resource usage as to fits to the business actual uses.

Third would be making the visualisation based on the recorded position of each object as a coordinate point to be drawn on the overlay of the video and based on it a path of movement alongside a density heatmap would be produced. Kalman filter smoothing is being used to correct the error and remove outlier that was produced by the network. The filter is also being used to average out the measurement from the network. This will result in smooth path lines that are otherwise impossible without any smoothing. Lastly, the result of the filtering would be feed into a sort and visualisation mechanism to draw an overlay imposed on a static image of the first frame of the video to make a visualisation output to be ready to be used and analysed.

Research Result


The model shows primary data enrichment and automation in which more can be developed to enhance usability. This current study offers a way to do it in much smarter, easier way rather than manually tracking each customer movement or their path, by outsourcing the task to a machine algorithm, it able to do so much faster and quicker with consistency, thus a model were proposed and evaluated for its output.

The blueprint layout ot the target store, with dashed line as camera view

It proved capable of supplying input into the design of the store layout. This is where the visualization and quantification of user location, intensity and pattern play by visualizing human movement and position. Managers and decision-makers would be able to observe their company in terms of store layout, customer contact with them, and how their product placement performs, future or actual issue with it, and how best they can do it by taking the interpretation and analysis result coupled with their existing data about store floorplan and product placement to create a full story. The story about how their customer interacts with the store, the products and fellow customer and their pattern. This in turn would be a valuable knowledge to act upon.

Conclusion

This research is limited in basic technicality and data gathering. It explores what the initial water test would be before going knee-deep into big data visualizations of retail and related premises data.

This research limitations are that it uses experimental method and not assessed in actual, real world or real time settings yet but use methodology akin to a lab simulating its theory and model. Conducting a field experimentation would be a truer test of the model proposed. This research would suggest that more data would be beneficial to be incorporated into the mix as well as a supporting method such as associative market basket and whole layout mapping. This research would suggest businesses to apply big data into their business processes and a simple model such as proposed and evaluated in this research would be a good way to start having a wet toes in data enriching world.

This project was done in a way that is very technical in nature for a business student, and is a culmination of myself as a researcher in a big data lab, which is a field usually reserved for computer science student. I learned a lot of techniques by trial and error and doing, and gain understanding on how to get an edge by doing techniques and exploiting what is avaiable (Such as imaging data in this case)

Overall, I am pleased of how this turned out, want to expand more if there is time and means available and it was a very good learning experience.

This research was presented at Annual Applied Science and Engineering Conference 2020

Back to top
EmailLinkedin

I am avaiable to contact and connect! Feel free to do so, I love to hear opportunities and share thoughts