Final Remarks

Project Summary

We’ve created a two part system to allow DJI Phantom 3 drones to be used for parking enforcement. The first component, the ‘Patrol’ Android application allows the Phantom 3 send images (along with GPS, and timestamp) to our server, which is the 2nd component of this project. The server runs license plate recognition, data aggregation upon the results, applies enforcement logic, and finally runs a web interface for users to view and process citations. Our system has been made robust with authentication to ensure only approved drones can submit evidence to the system.

Users are able to access our user interface to view and process citations at: http://taglabrouter.genomecenter.ucdavis.edu/webservice/

Resources:

1. All project code can be found at: https://github.com/quadsquad193/phantomboreas
2. User manual can be found at: https://www.dropbox.com/s/syu4eotxcqmv218/QuadcopterUserManual.pdf?dl=0
3. Patrol Android application apk could be found at: https://www.dropbox.com/s/oodbwicxhjcack4/app-debug.apk?dl=0
4. Project Overview Video: https://www.youtube.com/watch?v=6PEZUbAusp0

 

Thanks again to Professor Tagkopoulos and Professor Liu for their guidance and support.

-Baotuan, Kelvin, Mark, and Alex

Mobile: GPS/Timestamps & Next Up!

We successfully implemented GPS and Timestamp retrieval. We are now able to send this data with the pictures of the license plates to the server. The server now has the tools to implement business logic regarding actual time-based parking enforcement!

Next up for the mobile app is implementing authentication so unauthorized users cannot simply download the apk and send off pictures. The mobile and server teams will be working largely in concert with one another for this.

Lastly, we are about 3 weeks from a preliminary delivery to our clients so stay tuned!

Spring Break Update: Autonomous Flight

I have been spending the spring break exploring DJI’s Android Waypoints API for autonomous control of the quadcopter. The API allows us to execute a custom mission by creating ‘MissionSteps’ and inserting them into a queue to be processed sequentially by the quadcopter. MissionSteps are simple actions such as taking off, landing,  gimbal manipulation, taking a picture, yaw of the quadcopter, and the Waypoints feature, which involves setting a GPS coordinate and altitude to which the quadcopter will automatically fly. We originally intended to map out a simple scenario in a parking lot for a demo of our system. The quadcopter would fly on its own to predetermined parking spots, take a picture, and send it off for processing on our server. However, we have run into much difficulty in developing with Waypoints. The API is poorly documented, with limited description of the behavior of the different MissionSteps. I had to try many of the mission steps just to discover the exact behavior. This has proven rather dangerous. In fact, early today, while testing, the quadcopter suddenly took off and crashed into my house. Fortunately, I was able to catch the drone as it fell. The only damage to the drone was a broken propeller. I have ordered replacements and they should arrive by this Sunday. After this experience, I question the safety and viability of pursuing an autonomous system. We shall discuss with Professor Tagkopoulos at our next meeting on how we should proceed on this portion of the project.

Android App: Half-way mark

We made it to the end of Winter Quarter with a successful demo to our professor.

We were able to successfully control the drone, take a picture onto the drone’s SD card, download that image onto the mobile phone’s storage, and upload it to a server via an HTTP Post.

Picture Taking

We utilized the DJI SDK API and expanded upon it to save an image onto a known location for further processing. We encountered many difficulties with establishing the connection, but it’s working now.

Picture Transfer

The Android App successfully runs a media scanner on the DJI Phantom 3’s SD card and transfers it to the Android App.

Picture Upload

The Android App successfully uploads the image to a server via an HTTP Post.

Server

Currently the server is an Ad-Hoc network setup from one of our computers. It reads the incoming image feed, processes it, and performs database analysis and storage. Post analysis of the image, the server stores the image itself, the captured license plate, the latitude/longitude, and the timestamp of the image.

Known Issues

• The captured image isn’t indexed by the Android device, but we confirmed that it indeed gets stored onto the phone storage. We can upload the image from that known not-indexed location. Because of this indexing issue, you can’t see the image in the gallery, but you can see it with more robust file explorer apps.
• We currently don’t query the DJI Phantom 3 for the longitude & latitude of the captured image’s location. We also currently don’t post the timestamp of the image. Currently those are hard-coded, but will later be fixed to reflect actual coordinates and timestamps.

Team Progress Update (2.29.16)

Our goals for this week are to complete communication for a photo taken from the DJI quadcopter to be sent to a server and be processed by OpenALPR to obtain the license plate reading result.

What’s done so far:

1. Kelvin has been working on the backend for the server, which is now able to accept an image from a multipart post, and obtain the OpenALPR results.The whole backend consists of three processes and a Redis store:

   1. `droneservice` is a thin Flask application that accepts a post request and puts information onto a job queue in Redis.

   2. `openalprservice` consumes images from Redis and uses the OpenALPR library to do recognition. It then reports any results back into a Redis “results” queue.

   3. `parkinglogservice` ingests results from Redis and will do something with it. This is where the business logic will go. For now it just prints out OpenALPR’s license plate recognition results. We are at the point where we need to discuss more specific business details and begin to implement `parkinglogservice` to do meaningful work.

 

2. Android to server communication. Alex has completed a demo app to be able to send a photo from Android gallery to a server using a multipart post. We have yet to try this functionality with Kelvin’s recently completed backend.

3. Android application to control camera and retrieve photos. We are having trouble with this for the reasons described below. At this point, we are unable to make any further progress until the issue is resolved.

What is affecting our progress (urgent!):  We are currently having trouble connecting to the DJI quadcopter with our Android application. We believe it is due to the out of date firmware on the drone as well as the controller. We are unable to make the update however, since the drone battery level is too low and we do not have the correct charger. We need to be able to charge the drone ASAP to continue working.