In the past few years, the world has begun returning to its pre-COVID pandemic state - especially in terms of physical events. From ComicCons to Tech Conventions, huge physical events are back and in full swing and organisers need to prioritise security for those in attendance.

Despite the advancement of GPS technology, it is still difficult to accurately track locations indoors - especially when huge events are taking place. This is due to buildings being prone to blocking and weakening GPS signals with their dense construction materials. In addition, how can we accurately track people’s movements in locations like convention centres while respecting the privacy of individual citizens? In this article, Professor of Computer Science Rajesh Balan, discusses his research and development of DenseTrack technology and its potential applications in the real world.

The science behind the vision

Q: What is DenseTrack exactly?

Prof Balan: DenseTrack comes from the observation that in very crowded cities, like Singapore, it's actually very hard to locate someone moving through a building. One theoretical solution is to put cameras everywhere, but that's highly invasive and also very expensive. Now why is it important to be able to track people? It's about safety. With so many things going on and with all the crowds, you want to be able to know where people are in your building. If you need to evacuate certain areas, or if you need to identify people who might be up to no good. So this actually has a lot of safety implications, especially in dense urban environments.

So how do you do this? What my PhD student Truong Quang Hai and I realised was that if you want to track people on such scales, you need two things: something to identify where they are and something to identify when they move.

We use the Wi-Fi signal to track movement. A Wi-Fi signal is really good at identifying when someone’s moved from one side of the building to another. The issue becomes that Wi-Fi signals indoors are actually very inaccurate and cannot be used to accurately locate people, especially when there's a lot of people.

So to locate people we actually use cameras. Since cameras are not always all over a building, we identify the person using the Wi-Fi signal as an ID. That's the idea of DenseTrack.

How DenseTrack works:

• Leveraging Existing Infrastructure: DenseTrack uses Wi-Fi signals from existing enterprise network deployments and video from surveillance cameras.
• Wi-Fi for Unique Identification: Wi-Fi MAC addresses are used as unique identifiers for devices. Even with MAC address randomisation, a connected MAC address persists long enough for tracking within a building.
• Video for Location: Computer vision algorithms analyse video data to detect and locate people in each camera's view.
• Combining Wi-Fi and Video: The core idea is to match Wi-Fi MAC addresses to the video blobs. This combines the unique identification capability of Wi-Fi with the location information from video.
• Addressing Wi-Fi Limitations: Wi-Fi localisation is not accurate in dense environments due to interference and multipath effects. DenseTrack uses Wi-Fi as a coarse indicator of presence rather than relying on precise Wi-Fi-based location.
• CAIVU Algorithm: The "inCremental Association of Independent Variables under Uncertainty" (CAIVU) algorithm is at the heart of DenseTrack. It's inspired by the multi-armed bandit model and is designed to handle the uncertainties and complexities of real-world environments.

To simplify, computer vision algorithms identify people in each camera's view and assign them temporary IDs. The system then monitors Wi-Fi signals to detect connected devices (identified by their MAC addresses). The CAIVU algorithm attempts to match Wi-Fi MAC addresses to the video based on factors like proximity, movement patterns, and connectivity information. Once a match is made, the system can track a person as they move from one camera's view to another, even if the cameras don't overlap.

We managed to test this using one of the hardest scenarios you can imagine. A large exhibition hall in Singapore. It was during one of the largest pre-COVID events with 800,000 people attending over three days. We used the technology to track people moving around from floor to floor. In the end, the accuracy was either exact or one or two people off. When you're talking about thousands of people, that's actually really good. We either located the person exactly or the person next to the person.

Click to interact

Q: What makes DenseTrack a better option than available and already existing counterparts?

Prof Balan: DenseTrack is optimised for the business owner because they don’t need customers to do anything. The cameras and Wi-Fi are owned by the business owner, so there's no active participation by the people in the mall. So, if security gets a report that somebody on level one was showing suspicious behaviour, they can figure out where they are in the building immediately and be more precise in their response.

Q: A concern raised regarding tracking technology by people is the breaching of privacy, how can DenseTrack ensure that no private information is breached?

Prof Balan: The data collected by DenseTrack is public data. Once someone enters a building, they’ve given implicit consent to the building owner to be seen and tracked.

Now these businesses can't interact with customers unless they’re given more consent. So in this sense privacy is preserved as building owners still get the security they require but at the same time they can't upsell the visitors or spam them because they have no interaction channel to use to contact them.

Reframing the future

Q: If this technology were fully realised and made widely available, how would you see it affecting different industries and society as a whole?

Prof Balan: I think DenseTrack is a foundational building block. If you think about the future. It allows people who manage big buildings to know exactly where everyone is in the building without needing to spend money to build or install new physical components.

So what DenseTrack makes possible in the future, is that more people can track who enters their buildings. Building wonders and operators may not know who they are, but they'll know where they are now.

For example, if somebody leaves a bag somewhere and it’s thought to be suspicious. Later security can locate the video footage and reverse the person's movements to retrace all their steps since they’d know exactly where they went. So in this sense it would make society safer.

Q: Would the technology have a different impact or application in urban areas vs rural areas?

Prof Balan: This solution was actually developed for a country like Singapore where these sensors are already available. It turns out that, for various reasons, buildings already have lots of cameras and Wi-Fi. In areas which are less built-up, if they don't have cameras or WiFi, it's unclear if  this solution will work.

Challenges for implementation

Q: From your perspective, what are the key challenges or obstacles that we would need to overcome for this to manifest?

Prof Balan: There are two things. First, there needs to be sufficient video coverage in the building. It doesn't have to be comprehensive, but it has to be sufficient. Second, there needs to be some kind of signal that connects people for tracking their movement through the building. In Singapore we use Wi-Fi but it could be any other signal in other places, RFIDs could work.