Abstract
This paper describes a low computation system for a surveying service robot in a banquet hall setting. Automating this process will allow staff to focus on other tasks which will improve the overall efficiency of the event setup. The robot platform used was FURO, a two-wheeled service robot with an onboard computer as well as a camera. Image processing algorithms were developed using the open-source Open CV library to detect a total item count and ellipse tracker program. Potential fields were the navigation algorithm implemented to generate a collision-free path towards the edge of the table. The results were successful as the computer vision algorithms worked 90% of the time in a controlled environment with a normal setup. It experienced failure when items were overlapping or too close to each other. Differences in lighting also came into effect when detecting the contents of the table. Nonetheless, this experiment presents a service robot application for table monitoring in a ballroom.
-
Author Keywords
- Service Robotics ,
- Path Planning ,
- Navigation ,
- Computer Vision
-
IEEE Keywords
- Service robots ,
- Mobile robots ,
- Navigation ,
- Computer vision ,
- Cameras ,
- Image edge detection
Introduction
Over the last decade, service robots have been higher are present. Hence, the robot processes the contents; if the inspect tables in a banquet hall and ensure that all the utensils paper, the humanoid service robot shown in Fig. 1 is used as offices and homes as they can interact with tools. In this allows them to strive in human-centered environments much resembling the appearance of a human. Their human form is an option as they value the human-robot interaction by the environment they are placed in. Humanoid service robots interact, they must have designated functions that enhance order to successfully implement service robots with human extremely complicated and may experience malfunctions. In much more convenient. However, these service robots can beby Starship Technologies which make ordering food that cuts objects with great precision, to food delivery robots industrial KUKA robot arms in which a robotic arm weldsdemand due to their versatility in several situations. From the table setting is not complete, the robot will flag that specific table. This notification will be transferred to the staff to resolve the issue efficiently. With hundreds of tables in a banquet hall, there are bound to be errors during setup. The main purpose of this application is to increase service speed and allow human servers to facilitate customer satisfaction. This application is similar to retail monitoring robots which detect out of stock situations [1]. Yet, most of these methods apply a machine learning algorithm that requires high processing power. The platform used does not include a high end graphics processing unit, therefore the proposed method is a low processing method for filtering images and detecting contents on a table. The paper is organized in the following form: Section describes the related work in the service robotics and product counting applications, Section III describes the approach to solve the problem, Section IV presents the results of the experiment, and Section V will conclude the paper and prevent future work.
Theapplications of service robotics range from educational,entertainment, household, social, gaming, and research [2]. Yetthe project presented resembles a home application as it isprogrammed exclusively for a single task to be repeated. Thisis seen in the retail industry as mobile robots are deployedto detect out of stock situations. Temporary out of stock isa serious issue in the retail industry as it costs businesses4% of their revenue yearly. Either the shelf is not beingreplenished fast enough or the manufacturer is making an error.It is estimated that around 70%-90% of the problems is dueto malpractices regarding shelf replenishment [1]. One wayothers have approached the problem is to attach a monocularcamera on a mobile robot. This robot will navigate around thestorage room accessing its current inventory. It includes an onboard versatile camera that can adapt to the shelf height as wellas provide data for an object identification algorithm. Kejriwalet. al proposes fitting bounding boxes and segmenting themfrom the future frame sequentially [3]. Detection methodsutilize Speed-Up Robust Features (SURF) to find keypointsbetween frames that make up a k-d tree in the feature space [4].Other object detection methods are presented by Zimmerman[5] which decodes a product barcode from the image. Thisimage is sent to the database where it compares the barcodeto several other similar ones. Conversely, Gokturk [6] utilizescameras and lighting methods to find the number of itemsusing triangulation methods. Depth sensors and a stereo-visionsystem are recommended to obtain accurate results, yet onlya web camera is used in this paper to provide a low cost solution. Similarly, Kejriwal et. al [3] only use a camera as it addresses the issue of obtaining accurate results from just a photograph. The experimental methods presented in [3]can be divided into two steps. First, the product category is labeled, and secondly, the number of that product is estimated. Both of these results are obtained using two testing methods. The first method involves computing feature repeatability for each product which counts the maximum number of times a particular feature is repeated in a given image. The second method uses homography to make a 3 by 3 grid around the object. Furthermore, the second method not only gives the product count, but it also gives the product arrangement. In the application proposed, product count is the main focus which will be done by using image processing techniques on the incoming video frame.
Another approach to the manage inventory and stock in retail stores is to incorporate image sensors efficiently. Several patents have tackled the problem and developed units to address the current content available as well as out of stock situations. Birch proposes using a micro digital signage (MDS)unit which can take pictures of videos [7]. The MDS unit can also include a motion detector, a heat detector, and aGPS, which all contribute to finding and securing a product. The MDS unit takes a picture at the beginning of the day and another during peak hours. These two images are then compared by the MDS unit and it is determined whether the product needs to be restocked, signaling the employee to restock the product. Comparably, the experiment in this paper proposes to generate a comparison between the actual number of plates that are supposed to be on the table to the live video data received by the robot. A signal will also be sent to the staff to resolve the issue immediately. Groenevelt demonstrates a similar idea in his patent, but the system used includes an image capturing device. This device receives the image and determines whether the product needs to be restocked [8]. The patent also suggests using closed-circuit television (CCTV)as a capture device as it is common among retailers. Object recognition analysis is based on the training images and visual feedback from the system. One method includes creating a planogram and comparing it to a targeted planogram. The use of training images for object recognition requires high computation. In the experiment presented in this paper, the solution is a low computation applied to the problem of surveillance and autonomous navigation.
The use of robotics continues being a feasible method for inventory surveillance as there have been several patients resolving this problem. Zimmerman presents a mobile robot that traverses around a store and identifies items on shelves, a location for items, and a bar code [5]. The robot performs inventory checks by going through the store via inventory maps, capturing shelf image, decoding a product’s bar code or identification purposes, and determining whether an item matches the retrieved image. This concept works by generating a product database comprising of items, location of items, and the bar code for each of the items. To generate the inventory map, the robot reads the tracking tags and waypoints placed on the ceiling tiles, beams, poles, and shelves of the store; the purpose of these tracking tags is for the robot to avoid a collision with its surroundings. The navigation system described in [5] resembles the potential field algorithm created in this experiment as it has a predefined location of the obstacle positions. Similarly, Kumar [9] proposes a teleoperated robot for the same purpose of reducing manpower and increasing efficiency of detecting out of stock situations. Yet teleoperationstill requires a human in the loop therefore an autonomous system serves more benefit for efficiency.
Conclusion
We presented an experiment that validates the potential field and computer vision algorithm developed for a service robot. This is a low processing solution as no high-end GPUs needed to run the ellipse and counter algorithms. The algorithms were created using Open CV and integrated into theROS environment to communicate with the robot. The FUROrobot was the testing platform that successfully navigated a table and detected the contents of the table, verified complete or incomplete setting, and published data on an OpenCV map to notify the staff that the table is missing an item or complete. Future work will consist of expanding the algorithm to consider a series of tables as well as having it navigate and detect simultaneously. Another option is to improve the image processing techniques to deal with different lighting and environments as it will simulate an uncontrolled environment. Glare from the floor has to be taken into account as well as the reflections from the plates and utensils. In the future, FURO can also be used for multiple applications in a banquet-hall such as asking for feedback through the interactive touchscreen panel throughout the event as well as taking pictures to document the experience.
About KSRA
The Kavian Scientific Research Association (KSRA) is a non-profit research organization to provide research / educational services in December 2013. The members of the community had formed a virtual group on the Viber social network. The core of the Kavian Scientific Association was formed with these members as founders. These individuals, led by Professor Siavosh Kaviani, decided to launch a scientific / research association with an emphasis on education.
KSRA research association, as a non-profit research firm, is committed to providing research services in the field of knowledge. The main beneficiaries of this association are public or private knowledge-based companies, students, researchers, researchers, professors, universities, and industrial and semi-industrial centers around the world.
Our main services Based on Education for all Spectrum people in the world. We want to make an integration between researches and educations. We believe education is the main right of Human beings. So our services should be concentrated on inclusive education.
The KSRA team partners with local under-served communities around the world to improve the access to and quality of knowledge based on education, amplify and augment learning programs where they exist, and create new opportunities for e-learning where traditional education systems are lacking or non-existent.
FULL Paper PDF file:
Service Robot Navigation and Computer VisionApplication in a Banquet Hall SettingService Robot Navigation and Computer VisionApplication in a Banquet Hall SettingBibliography
author
Year
2020
Title
Service Robot Navigation and Computer Vision Application in a Banquet Hall Setting
Publish in
2020 10th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA, 2020, pp. 0918-0923,
Doi
10.1109/CCWC47524.2020.9031123.
PDF reference and original file: Click here
Somayeh Nosrati was born in 1982 in Tehran. She holds a Master's degree in artificial intelligence from Khatam University of Tehran.
Professor Siavosh Kaviani was born in 1961 in Tehran. He had a professorship. He holds a Ph.D. in Software Engineering from the QL University of Software Development Methodology and an honorary Ph.D. from the University of Chelsea.
Nasim Gazerani was born in 1983 in Arak. She holds a Master's degree in Software Engineering from UM University of Malaysia.
