Final Remarks

Project Summary

We’ve created a two part system to allow DJI Phantom 3 drones to be used for parking enforcement. The first component, the ‘Patrol’ Android application allows the Phantom 3 send images (along with GPS, and timestamp) to our server, which is the 2nd component of this project. The server runs license plate recognition, data aggregation upon the results, applies enforcement logic, and finally runs a web interface for users to view and process citations. Our system has been made robust with authentication to ensure only approved drones can submit evidence to the system.

Users are able to access our user interface to view and process citations at: http://taglabrouter.genomecenter.ucdavis.edu/webservice/

Resources:

1. All project code can be found at: https://github.com/quadsquad193/phantomboreas
2. User manual can be found at: https://www.dropbox.com/s/syu4eotxcqmv218/QuadcopterUserManual.pdf?dl=0
3. Patrol Android application apk could be found at: https://www.dropbox.com/s/oodbwicxhjcack4/app-debug.apk?dl=0
4. Project Overview Video: https://www.youtube.com/watch?v=6PEZUbAusp0

 

Thanks again to Professor Tagkopoulos and Professor Liu for their guidance and support.

-Baotuan, Kelvin, Mark, and Alex

Team Progress Update (2.29.16)

Our goals for this week are to complete communication for a photo taken from the DJI quadcopter to be sent to a server and be processed by OpenALPR to obtain the license plate reading result.

What’s done so far:

1. Kelvin has been working on the backend for the server, which is now able to accept an image from a multipart post, and obtain the OpenALPR results.The whole backend consists of three processes and a Redis store:

   1. `droneservice` is a thin Flask application that accepts a post request and puts information onto a job queue in Redis.

   2. `openalprservice` consumes images from Redis and uses the OpenALPR library to do recognition. It then reports any results back into a Redis “results” queue.

   3. `parkinglogservice` ingests results from Redis and will do something with it. This is where the business logic will go. For now it just prints out OpenALPR’s license plate recognition results. We are at the point where we need to discuss more specific business details and begin to implement `parkinglogservice` to do meaningful work.

 

2. Android to server communication. Alex has completed a demo app to be able to send a photo from Android gallery to a server using a multipart post. We have yet to try this functionality with Kelvin’s recently completed backend.

3. Android application to control camera and retrieve photos. We are having trouble with this for the reasons described below. At this point, we are unable to make any further progress until the issue is resolved.

What is affecting our progress (urgent!):  We are currently having trouble connecting to the DJI quadcopter with our Android application. We believe it is due to the out of date firmware on the drone as well as the controller. We are unable to make the update however, since the drone battery level is too low and we do not have the correct charger. We need to be able to charge the drone ASAP to continue working.

Update from Team Meeting 2.20.16

After meeting with Professor Tagkopoulos on Friday, we have reprioritized working on the image recognition task. We have decided upon using the OpenALPR project, a well-developed open source project, to supply the computer vision and image processing capabilities for our project. OpenALPR provides many pre-built web services and daemons to perform license plate recognition tasks, as well as an integration API with numerous language bindings. We have tested this functionality and it works reasonably well upon some sample images of vehicles that we have tried.

Here’s the link to the project: https://github.com/openalpr/openalpr

Here’s the sample output we got:

To be able to integrate the drone, we need a means to store images from Quadcopter to server for processing by OpenALPR. The previous team has implemented this for Google glass. We will need the drone and the code from last year to begin testing and optimizing the recognition for pictures taken from the drone.

Goals for the week:

• Be able to process images with OpenALPR from server. Have a working demo to show Professor Liu and Professor Tagkopoulos.
• Delving deeper into last year’s code concerning DJI control and DB communication to determine what we could reuse. Daniel Chen from last year’s team has given us access to the BitBucket repository for their code.

 

Looking ahead, we may need to develop a custom android app using parts of last year’s design if we want autonomous control with DJI’s waypoints. Also, if we are able to have a reasonable working demo by end of this quarter with OpenALPR, we may be able to pursue more advanced recognition methods. These are, of course, “nice to have” features that would be planned for next quarter but are not a priority right now.

Tentative Plans for Infastructure

This week, our group met and discussed potential plans for the general infrastructure of our project. Given that the only programmatic interface provided with the DJI Phantom 3 Advanced is a Mobile SDK, the first level of communication from the DJI Phantom 3 will be through an Android phone. At this point, our plan is to send select images from the quadcopter to the phone instead of a live feed, as a live feed would provide far more data than we need.

We then plan on wiring the phone to a laptop via a USB connection to programmatically grab the images from the phone as they arrive. The laptop will run the necessary segmentation and OCR to find and read licence plates and will track that info along with other contextual info in a local SQL database.

We tentatively plan to model our licence-plate-reading framework based on this recent paper.  When an overparked car is found, the mobile application will receive a notification with the relevent information — notably location, picture, and license plate data.