Computer Vision in Restaurants: 5 Use Cases

Restaurants simultaneously represent one of the longest-enduring and one of the fastest-changing industries. At the same time that St. Peter Stiftskulinarium in Austria welcomes guests to the second-oldest extant business in the world, Mezli in San Francisco uses robots to serve its customers in the world’s first fully-autonomous restaurant. 

While restaurateurs have always been on the lookout for new technologies, the COVID-19 pandemic has served as a catalyst for innovation. During the height of the pandemic, drive-thru ordering and curbside pickup surged, which left many restaurants struggling to keep up with demand, especially given that restaurant staffing was historically low. 

Unfortunately, this strain often caused a drop in customer satisfaction due to reduced service quality or increased wait times. While staffing issues have improved, restaurants still have not reached pre-pandemic staffing levels, causing problems to persist. 

Between more frequent order mixups, exceedingly long wait times, increased consumer concern about hygiene, and staffing shortages, modern restaurateurs have a large host of new challenges to work through. 

Computer vision is one emerging technology that restaurants have turned to as a solution. This AI-powered tech can help improve order accuracy, reduce wait times through increased efficiency, elevate customer service, ensure safety and hygiene measures are adhered to, better direct marketing efforts, and optimize restaurant operations. 

Get started with live wait (queue) time monitoring.

What Is Computer Vision?

Computer vision (CV) is a field of artificial intelligence (AI) that seeks to leverage the raw processing power of computers to interpret the visual world. In other words, computer vision technology tries to get computers to “see” the world so that they can draw insights and take actions based on visual data, just like we humans do. 

For example, computer vision can be used for facial recognition, autonomous driving, or even to power a Roomba. In the commercial and industrial sphere, it’s often used in retail, construction, traffic and parking, and the hospitality industry. 

Computer vision works by taking a visual data source, such as an image, video, or real-time video stream, and converting it into a form that a computer can analyze. From there, the computer uses deep learning, machine learning, convolutional neural networks, and various algorithms to understand what it’s seeing. 

The end result is, ideally, a level of consistently accurate, sustained, and insightful visual interpretation that humans are simply unable to achieve due to physiological limitations. For example, a single human will struggle to accurately monitor all the people that pass by a restaurant on a busy street over an eight-hour period to calculate the percentage that come inside, but a computer can do so easily. 

5 Use Cases for Computer Vision in Restaurants 

Like many business owners, restaurateurs have seen the potential of computer vision and have started applying it to their operations. Use cases include:

• Order accuracy
• Drive-thru and operations efficiency
• Hygiene and safety guideline adherence
• Marketing and consumer interest analysis
• Customer service

Order Accuracy

In 2021, the American Customer Satisfaction Index (ACSI) found that a whopping 26% of fast-food orders were inaccurate. This represented an increase in mistakes compared to the prior year, largely driven by the surge in drive-thru orders at quick-service restaurants (QSR) and fast-food restaurants due to the pandemic.

Inaccurate order fulfillment can pose serious problems to restaurants and consumers. Not only are mixed up orders annoying to customers, but in rare cases, serving up the wrong order can lead to serious injury, death, and lawsuits — if your waitstaff mistakenly serves peanut sauce to someone with a severe allergy, the consequences can be dire. On a less catastrophic scale, serving the wrong food can seriously offend a patron’s moral sensibilities — for example, serving a meat dish to someone who ordered the vegan version. 

Whether the immediate result is a mildly irritated customer or a hospitalized patron, the long-term consequence can be a once-loyal customer who decides to take their business elsewhere. 

Thankfully, order accuracy is one of the areas where computer vision shines. One commercial CV company found that their technology could spot 85% of all order inaccuracies and fix the problem before the food was ever served to customers. Based on the ACSI’s figures, this system could bring the error rate down from 26% to just 3.9% of all orders. 

Drive-thru and Operations Efficiency

During the COVID-19 pandemic, the restaurant industry saw a huge increase in drive-thru and takeout interest. Although consumer interest in indoor dining has returned and restaurants are starting to achieve pre-pandemic levels of revenue, restaurateurs should expect drive-thru options to remain popular both due to newly formed habits and a sizeable portion of the population that isn’t yet comfortable with indoor dining (25% of respondents to Morning Consult’s June 2022 survey indicated they are not comfortable eating indoors). 

Computer vision can help manage this increased demand in two ways: restaurant preparation and customer expectation management. 

With computer vision tools, restaurants can analyze traffic patterns within the drive-thru over time to accurately determine when more staff are needed. However, CV’s utility isn’t strictly limited to historical analysis: computer vision can be used in real time to count the number of cars currently in the drive-thru and predict how many will order burgers and how many will order fries based on past data. With this information, the kitchen can immediately begin appropriate preparations without delay as well as better prepare operationally for the busiest hours of the day and week. 

Additionally, CV systems can detect when customers abandon lines to help pinpoint issues and optimize restaurant operations. 

On the customer side, computer vision can predict wait times and display them to patrons in line to manage customer expectations — customers are more likely to be dissatisfied if their wait is longer than anticipated, and CV can help make more accurate predictions to improve customer experience. It can also monitor indoor table occupancy so that restaurants can strategically mobilize staff. 

Hygiene and Safety Guideline Adherence

Although mask mandates are no longer in effect in most jurisdictions, the pandemic has likely permanently heightened consumers’ awareness of hygiene, health, and safety issues. For restaurants to be successful in today’s world, they need to make their customers feel safe. 

Computer vision is being applied across several industries (perhaps most widely in the construction industry) to ensure worker safety. In the restaurant industry, computer vision systems can detect when restaurant workers aren’t wearing proper PPE, such as masks, hairnets, and gloves. As the technology advances, it may also be able to detect other safety violations, like cooked food being placed in a raw food area or food that is left out of the refrigerator for more than three hours. 

In all cases, catching these mistakes can help maintain consumer confidence and reduce the likelihood of food-borne illness outbreaks, which can irreparably damage a restaurant’s reputation and finances. 

Marketing and Consumer Interest Analysis

Alongside immediate improvements to efficiency, safety, and accuracy, computer vision can also be utilized to help make long-term strategic decisions. Retail businesses and event spaces, for example, are using computer vision to create detailed heatmaps that track where customers and attendees gravitate to within a store or event space. This data can then be analyzed to inform product placement and other strategic initiatives. 

Similar technologies can be used in restaurants. While heatmaps themselves are slightly less useful due to the sedentary nature of most dining experiences, gaze detection can be used to gauge customer interest in different offerings. These systems can be setup outside the restaurant to analyze where potential customers are looking within a window display or inside the restaurant to see what items they are most interested in. 

CV can also monitor the outside of a restaurant to see how many people that stop in front of the restaurant ultimately come inside. These results can then be paired with gaze detection to help determine if there is any relation between items that potential customers look at and whether they come inside. 

Customer Service

Although still in its fledgling state, some companies are looking into using computer vision to improve customer service by recognizing loyal customers and providing them with more personalized service. Additionally, computer vision can be used to evaluate staff engagement to better understand a key driver of customer satisfaction. 

Key Takeaways: Computer Vision for Restaurants

The COVID-19 pandemic caused major losses for restaurants between 2020 and 2022. Although the industry has largely recovered, staffing shortages still persist, and restaurants are struggling to maintain high-quality service while meeting customer demand.

Computer vision is a novel way that restaurants can improve their operations and the customer experience in spite of staffing shortages. Through improved analysis for long-term planning, optimized lines, and enhanced order monitoring, restaurants can give customers a satisfying and fulfilling experience. 

Learn more about live wait time monitoring. 

‍