Use Case
The overall scenario is about a baggage service for airport customers. The passengers do not have to care about their baggage, as this is picked up and delivered directly to the airport. The service should cover the delivery from the customer’s home to the airport baggage check. The first step of the scenario is that the customer has to fill in all data in the interface model. This model is then used to provide a contract. If the customer agrees with the contract details, it is fixed. The details from the interface model are transferred to the process model. This model is relevant for the schedule, which is as follows. The steps are all confirmed by using the model as an interface. The car (mBot) stops at the house of the customer. The customer has to load the baggage into the car. Afterwards the car drives to the robotic arm, which should represent the unloading station. The arm unloads the baggage and carries it to the scale. The weight of the baggage is measured. This measure is checked by referring to usual data. The related contract details and the weight are used to define the following action by using a rule-based system. In case the baggage is too heavy or there is another reason for an additional payment, the customer receives a message via the process model. The baggage is then lifted by the arm to the baggage drop area.
Smart Models
In my case, the models serve as execution interface, where the information of the human is inserted into the model elements. Then the models are transformed into machine readable and executable models that are used for scheduling the execution process. Therefore, the models are intuitive for humans to read and also machine processable.
Semantic Technologies
There are various semantic technologies that can be included in this service. As there is no scale available in the OMiLAB, I decided to used a sensor for scaling the baggage. Some sensors on the car a used additionally and also a rule engine was applied.
Rule Engine: The rule should decide, based on common data, the scale sensor and attribute data of the baggage, the further processing. There are two possible decisions. In case everything is alright, the customer gets a confirmation message in the model. On the other hand, the customer gets an error message, which indicates that special fees are necessary. As a rule engine, I used Drools. This business rules management system is available for Eclipse and can be combined with Java, ontologies and all other necessary programs and resources.
Weight Sensor: The self-built scale has a HX711 sensor, which delivers the data to the raspberry. As the raspberry could deal with python programs the scale service is written with python. The scale is directly fixed to the car and the weight is measured every time the car is loading another piece of baggage.
Execution of Conceptual Models
The conceptual models serve as user interface as well as execution trigger. They require some abstraction of the cyber physical systems regarding interfaces. Therefore, the cyber physical systems provide some web service interfaces, so that the model can interact with the robots by calling these web services. These service interfaces have a fixed structure. When this structure is called, the function is executed and a specific behaviour is shown by the cyber physical system.
Experiment
Architecture and Interactions
The architecture for the experiment is like shown in the picture below. It is important to notice that the architecture is tightly connected to the available devices in the OMiLAB. The robots provide web services for using their features, the model is used as an user interface and afterwards it is executing the process by running AdoScripts. Everything is connected over the OMiLAB network.
Assumptions
- one mBot
- Tomcat on mBot raspberry
- mBot in OMiLAB network reachable
- driving program deployed on mBot raspberry
- fully charged storage battery
- stable internet connection
- one way street with 10 houses
- arm reachable in OMiLAB network
- scale reachable in OMiLAB network
- Tomcat on scale raspberry
Device Setup
In the following, the three main devices, the car, the arm and the scale are described.
mBot: The mBot has two sensors, the line following sensor and the ultrasonic distance sensor. The mBlock core board is used and normal rechargeable batteries are in the battery box. Two raspberries and the storage battery a fixed on the top of the mBot. The storage battery serves mBot and scale raspberry.
Scale: The scale consists of a load cell and a HX711 sensor. The construction is fixed on the back of the mBot car and connected to the scale raspberry via a breadboard and jumper cables. A surface for scaling was built up by using leftover hardware parts from the mBot.
Arm: The robotic arm from the OMiLAB was used as it was before. The configurations from the raspberry were not changed and the services were used via web connection.
Environment Setup
A street model and some baggage parts were used additionally. The street model was printed on normal paper, but it may be reasonable to use some paper that is more stable (eg: 160g). The elements were cut and glued together. Some tiny and light weight paper boxes were used to simulate the baggage.
Important Web Service Commands
In the following list, the methods used by the model are explained in short. Some additional services are for support, like checking the port numbers.
- status: gives distance of ultrasonic sensor
- moveStraight: car follows black line on street model
- turnLeft: right motor is activated until black line is available again
- turnRight: left motor is activated until black line is available again
- jumpGap: drives short distance to get next black line
- honk: sound output
- light: red and blue color output
- weight: scale gives back weight of load?
Deployment
To get the mBot ready for driving and for configuration, the power of the mBot must be plugged in first, then the power for the raspberries could be switched on. For the actual deployment a connection to the mBot must be established via Putty by using the correct IP address. It is essential to be connected to the OMiLAB network for the whole procedure. On both raspberries (scale and mBot), a software folder was created. In this folder the source code for scale or mBot and the tomcat can be found. In side the code folder, a git pull can load the updated code. But it should be the current version on the device. After a git pull, the project folder must be rebuilt by using mvn clean and mvn package afterwards. Then a .war file in the target folder is established. This file is copied to the webapps section in the tomcat folder.The tomcat was installed as service. It can be started and stopped via sudo service tomcat start.
It is recommended to try out the mBot with the original mBlock software and a simple Bluetooth connection beforehand, in order fo find out how they move and behave. The readme file in the code repository can be used for looking up further details. Not only the mBot, but also the scale should be used directly beforehand. This is done by running the scale.py programm with python directly and not using a web service. Necessary resetting processes can be executed easier this way. As a reference weight, some weight that is known should be used. In order to avoid the the car topples over backwards it is suggested to used small weights like a package of chewing gum for instance.
Domain Specific Modeling Language
The domain specific modeling language depends on two model types. The first one is the customer data model, that serves as customer interface. All relevant data for the process can be inserted. A script allows checking whether all necessary data was filled in. When everything is alright, the model turns green, otherwise the element, which is errorprone, gets red.
The second model type is about the process schedule. It depends on the customer data model entries and uses then as a foundation. It is possible to have up to three customer models. The process model is built accordingly and uses 1 to 3 houses and baggage. This model is then responsible for the execution and connection to the cyber physical systems.
Execution
There are four scripts that guide the execution. First of all, the modelling toolkit with the suitable modeling language must be opened. Then the first script can be used to create customer data models (1 to 3). Afterwards, the data in the elements must be adopted as wished. Moreover, the models can be checked with the second script. In case of green elements everything is correct, otherwise, some information for the process is missing. When this is done, the process schedule model can be created. In this model, only the ids of the car, the scale and the arm must be adopted (IP address or token). When all devices are turned on and ready, the process can be started by using the fourth script. Some additional scripts for testing might be useful.
Resources and Code
Especially the documentation files might be helpful for finding detailed comments on how the execution and the preparation of the scenario must be done. As the code was moved from my git repository to the official course repo, it might be necessary to update the url in the project folders on the Raspberries with the correct link (eg: git remote set-url origin git@gitlab.dke.univie.ac.at:ed
https://gitlab.dke.univie.ac.at/edu-semtech/baggageservice
https://gitlab.dke.univie.ac.at/edu-semtech/scaleservice
Results
Video
In this section, the results can be seen in a short example video. The film reveals the interactions between modeler, model and cps. Therefore, model and physical environment are executed in parallel. Only one simple successful scenario execution is shown in the video, for sure there are more possibilities.
Future Work
There are various possible extensions for the scenario. Below some of my ideas are listed and can be the basis for a future extension of the scenario.
Ontology: Ontologies can be used to classify whether the baggage is okay. This can be done via the weight and maybe another attribute like the color for instance. I would use either OWL or RDF, as both should be accessible via a Java program and the combination with the used rule engine should be of manageable effort. Also, the Apache Jena Framework, will be probably useful for combining the rule engine and the ontology.
Mecanum Wheel: The Mecanum Wheel robot could be used instead of the mBot. This would enable the loading of more than one baggage. The robot could drive from one customer to the next and when all customers loaded their baggage, the robot can drive to the unloading station.
RFID: By applying RFID tags to the baggage, the customer could track his own stuff, by reading out the RFID tag. A web service could be set up, that allows the readout for the customer online.
Image Recognition: Image recognition could be used for instance, as a mean of classification for the baggage. Furthermore, it could be used to recognize the different types of baggage on the Mecanum Wheel robot. As more than one baggage can be loaded on this, the arm should be informed about the coordinates via the image recognition and image detection technologies.
GPS: Also, a GPS sensor could be applied to the car, so that it is traceable in case of self-driving cars. Moreover, the GPS could be used to calculate the current, the customer and the robotic arm position coordinates to find out the best way.