Ducati Data Analysis - File Format - Want to Import with video - Ducati.ms - The Ultimate Ducati Forum
 
LinkBack Thread Tools Display Modes
post #1 of 5 (permalink) Old Oct 21st, 2010, 10:17 pm Thread Starter
Junior Member
 
Join Date: Oct 2010
Location: Oxford, England, UK
Posts: 2
Ducati Data Analysis - File Format - Want to Import with video

Does anyone know, or is able to reverse engineer, the format of the .DDA files?
Unlike alot of data analysis software the file format is not in binary format making converting and importing into another suite difficult.

The reason for this is that i want to use a different tool to analyse the data with then the plan to create a video with engine data from DDA (rpm speed etc) then use distance to plot position on a map overlayed onto pictures from the video/camera.

Any help would be much appreciated. I know buying a piece of equipment which can do all this is probably easier but as ive got all the necessary data i'd like to piece it together without spending more cash.

I have attached some sample data if anyone wants to play with the file. .CSV or .BIN would be what im looking for.

Many thanks,
Joe (England)
Attached Files
File Type: zip Run002-001999-12.15.dda.zip (186.4 KB, 257 views)

Last edited by thebatch; Oct 21st, 2010 at 11:22 pm.
thebatch is offline  
Sponsored Links
Advertisement
 
post #2 of 5 (permalink) Old Oct 22nd, 2010, 5:36 am
Senior Member
 
Join Date: Aug 2006
Location: Sydney, NSW, Australia
Posts: 331
Last year I reviewed the DDA file format, this was to enable a MacOS version of the DDA software. I have a Mac version as a result however time have been limited and I have not continued the development.

As a result below are my findings are in this "white paper". There are holes in the information and I cannot guarentee the accuracy of the data. Anyone who contnues to review the file format that can expand on some of the information, it would be greatly appreciated it you can pass the info back to me.


DDA Whitepaper (rev0)
January 2010

Introduction and General Notes:

The following is an account of my observations after reviewing the contents of a Ducati Data Analyser (DDA) file.

The described format appears true for files generated by the Ducati 1098, 1098S and D16RR models. It may or may not apply to other Ducati models (as more files are observed so this document will be updated).

Each DDA file consists of a header followed by the recorded data. The following data is recorded; engine speed, engine temperature, throttle position, vehicle speed, distance travelled and lap interval. The gear position is derived by dividing the engine speed by the vehicle speed.

The DDA device records the data at 100 hz (or in 0.01 second intervals), and to reduce data traffic (and to save space) only the relevant data is recorded at each time interval.

The data file contains no absolute or relative information relating to time or lap count, it is therefore up to the parser to determine the time applicable to a piece of recorded data and likewise keep track of the lap count.

Regardless of the measurement units set by the motorcycle user, all data is stored as metric units; temperature is stored in degrees Celsius, speed is km/h and distance is kilometres. If an output of another unit system is desired, then it is the parser that must conduct the unit conversions.

All data is stored in “little endian” format (For example the 16 bit hexadecimal value 01A5 (or decimal 421) is stored as A5 01).

This information is provided for your interest; however since I have no input into this data format and additionally no access or insight into the actual data specification or source code I cannot guarantee the accuracy of this information. Likewise as this is a privately controlled data format, it may be changed in the future thus rendering this information as obsolete.

Header:

So far two types header information have been identified, they appear to be listed as either be either version 2 or version 3 (that’s what this will document call them).

For Version 2 and Version 3 headers, the header data itself appears to be added by the application that downloads the information from the DDA dongle since there is no evidence of this data being stored on the DDA dongle.

Version 2 Header:

In this file type, the first 22 bytes are the header. It is identified by the initial hexadecimal characters “02 00” followed by the ASCII characters “DDA” as the first five bytes of the file (or “02 00 44 44 41”). The 22 header bytes are identical for each of the listed bikes and the author has not determined what other additional information is stored therein.

Version 3 Header:

In this file type, the first 296 bytes are the header. It is identified by the initial hexadecimal characters 03 00 followed by the ASCII characters “DDA” as the first five bytes of the file (or “03 00 44 44 41”).

This file type also reserves 2 x 64 byte spaces for the user to store the recording location and the riders name in the data file. Each text entry appears to be a NULL terminated ‘C’ string of ASCII characters. The location string commences at hexadecimal 21 and the name string at hexadecimal 61.

At hexadecimal A0 a 16-byte string exists that seems to determine the file type

0x00A0 – 4 bytes unknown significance – changes between runs
0x00A4 – 2 bytes appear to be “E8 C5” for DTS and “B5 B0” for non-DTS
0x00A6 – 2 bytes “CB 88” or “CC 88”: CB for DTS and CC for non-DTS?
0x00A8 – 4 bytes odometer reading for commencement of run
0x00AC – 4 bytes unknown significance

Samples of data at 0x00A4
6A C4 CC 88 – Desmo D16RR with race ECU (no DTC)
43 17 CC 88 –848 (no DTC)
EF C2 CB 88 – Streetfighter S? (DTC)
F8 47 CC 88 – 1198S (DTC)
B5 B0 CB 88 – 1098S (no DTC)

Speculation, does the DDA store the DTC level here?

The significance of the other characters is still undetermined.

Data Storage (Version 2 and Version 3 Non DTS files):

The following data is stored in the DDA file:

Engine Speed (Revolutions per Minute)
Engine Temperature (Degrees Celsius)
Vehicle Speed (km/h)
Distance (km)
Lap Button (100’s of a second)
Throttle Position (Percentage)

The data is stored in a tight packed formation with no external time references. It is therefore up to the parser to determine (or calculate) the time for any particular data element. The base unit however is 1/100 of a second (or 100 hz).

All data is stored in metric units and it is the responsibility of the host application to modify the data to the units of measurement preferred by the user.

Data Storage (Version 3 DTS files):

The following data is stored in the DDA file:

Engine Speed (Revolutions per Minute)
Engine Temperature (Degrees Celsius)
Vehicle Speed (km/h)
Distance (km)
Lap Button (100’s of a second)
Throttle Position (Percentage)
DTS Value (Percentage)

The data is stored in a tight packed formation with no external time references. It is therefore up to the parser to determine (or calculate) the time for any particular data element. The base unit however is 1/100 of a second (or 100 hz).

All data is stored in metric units and it is the responsibility of the host application to modify the data to the units of measurement preferred by the user.

Data Elements:

Engine Speed:

Revolutions Per Minute (RPM) – 16 bits every 0.02 seconds

Throttle Position:

Percentage Applied (%) – 8 bits every 0.05 seconds

Engine Temperature:

Degrees Celsius (deg) – 8 bits every 1.00 seconds
To calculate actual engine temperature subtract 40 from the stored value. This is the OBDII way of storing temperature such that negative temperatures can also be stored using only positive values (i.e. decimal value 74 corresponds to an engine temperature of 34 degrees Celsius).

Vehicle Speed:

Kilometres per Hour (km/h) – 16 bits every 0.10 seconds
The actual vehicle speed is determined by dividing the returned value by 4. As an aside, the OBD standard uses an 8-bit value to record speed, however this gives a maximum possible recorded speed of 255km/h and since the Ducati 1098 can exceed this speed a custom means of storing the vehicle speed was needed.

Distance Travelled:

Kilometres (km) – 24 bits every 1.00 seconds
Value is in whole kilometres (NB the figure could be 16 bits as every vehicle I’ve checked has had the value 00 for the final reading – however this would give a maximum odometer recording distance of 65,536 km, so a 24 bit value is logical).

Lap Button:

100ths of a second – 8 bits every 1.00 seconds
If the lap button is not pressed, the value FF is recorded.
If the lap button is pressed, the number of 1/100ths of a seconds following the previous second is recorded (e.g. if at the 56th second the hexadecimal number 12 is recorded for the lap byte, the lap button was pressed at 0”55’.18).
NB this means a maximum of only one lap can be recorded per second.

DTS:

Percentage Applied (%) – 8 bits every 0.05 seconds

Parsing the Data:

As indicated above the data is stored in a tight byte ordered structure with no spaces, checksums or absolute markers. Therefore the synchronisation of the parser to the data is critical and any corruption of the data file will likely result in a failed parsing of the data.

The DDA data stream contains no absolute time or clock data, however the data stream always start on a whole second (i.e. X’ XX.00”) and increments in 0.01 second intervals.

It makes sense that the parser nominally allocates a time of 0’00.00” (or zero) to the start of the data stream (this is not a requirement though)

Reading data:

As the data file is read the parser needs to increment time in 0.01 second intervals in synchronisation with the reading of data stream. As indicated above; no actual time information (or effective data check points) are recorded within the file.

The closest to a data checkpoint is the hexadecimal value FF that is stored once per second (if the lap button is not pressed). Using this marker as a basis might enable a corrupted data file to be partially recovered.

The data is stored in the order:

Speed -> RPM -> Temperature -> Throttle -> Lap -> Distance

Below is an unpacked data stream followed by the packed version of the same.

Time Bytes Data Included (size in bits):
0’00.00” 10 Speed (16), RPM (16), Temperature (8), Throttle (8), Lap (8), Distance (24)
0’00.01” 0 -
0’00.02” 2 RPM (16)
0’00.03” 0 -
0’00.04” 2 RPM (16)
0’00.05” 1 Throttle (8)
0’00.06” 2 RPM (16)
0’00.07” 0 -
0’00.08” 2 RPM (16)
0’00.09” 0 -
0’00.10” 5 Speed (16), RPM (16), Throttle (8)
etc.
0’00.95” 1 Throttle (8)
0’00.96” 2 RPM (16)
0’00.97” 0 -
0’00.98” 2 RPM (16)
0’00.99” 0 -
0’01.00” 10 Speed (16), RPM (16), Temperature (8), Throttle (8), Lap (8), Distance (24)

a bike travelling at 25 km/h at 3200rpm with an engine temperature of 87 deg Celsius and 4284 km on the odometer at a steady 20% throttle opening would record the following hexadecimal values in the first 0.10 seconds:

64 00 80 12 7F 14 FF BC
16 00 80 12 80 12 14 80
12 80 12 64 00 80 12 14
Andrew is offline  
post #3 of 5 (permalink) Old Apr 29th, 2015, 5:49 pm
Junior Member
 
bigbang's Avatar
 
Join Date: Apr 2015
Location: Charlotte, NC, USA
Posts: 3
Hi Andrew. Thanks a lot for this. I'm sure most people don't know how to use this information, but I found it very insightful. It has inspired me to do something with it.

I'm a novice programmer. My goal is to write a script that parses the data to a csv format for viewing logged data in a 3rd party motorsport software. I plan on using matlab to write this script.

I follow that basically once past the header, it just shoots out the signal values in little endian hex without a time stamp based on a known logging frequency and byte size for each message. So then I tried to understand your example...

Here are my questions:

1) So how do I get to hex? I've noticed the file is all gibberish if I open it in Notepad++, with a few recognizable words in the header. Is there some common code out there that converts the dda file to a stream of hex that I can chop down using a parser? Is it a basic function that I am overlooking?

2) I understand temp (-40), throttle, speed (/4) and the lap event, but I don't get distance and rpm. In your example:
-RPM is 80 12. In hex I get 12 80. In decimal I get 4736, but it should be 3,200?
-Distance is BC 16 00. In hex I flip that around to get 00 16 BC. In decimal that's 5820, but it should be 4284km?

Thanks

Quote:
Originally Posted by Andrew View Post

The data is stored in the order:

Speed -> RPM -> Temperature -> Throttle -> Lap -> Distance

Below is an unpacked data stream followed by the packed version of the same.

Time Bytes Data Included (size in bits):
0’00.00” 10 Speed (16), RPM (16), Temperature (8), Throttle (8), Lap (8), Distance (24)
0’00.01” 0 -
0’00.02” 2 RPM (16)
0’00.03” 0 -
0’00.04” 2 RPM (16)
0’00.05” 1 Throttle (8)
0’00.06” 2 RPM (16)
0’00.07” 0 -
0’00.08” 2 RPM (16)
0’00.09” 0 -
0’00.10” 5 Speed (16), RPM (16), Throttle (8)
etc.
0’00.95” 1 Throttle (8)
0’00.96” 2 RPM (16)
0’00.97” 0 -
0’00.98” 2 RPM (16)
0’00.99” 0 -
0’01.00” 10 Speed (16), RPM (16), Temperature (8), Throttle (8), Lap (8), Distance (24)

a bike travelling at 25 km/h at 3200rpm with an engine temperature of 87 deg Celsius and 4284 km on the odometer at a steady 20% throttle opening would record the following hexadecimal values in the first 0.10 seconds:

64 00 80 12 7F 14 FF BC
16 00 80 12 80 12 14 80
12 80 12 64 00 80 12 14
bigbang is offline  
post #4 of 5 (permalink) Old Apr 29th, 2015, 11:59 pm
Senior Member
 
Join Date: Oct 2014
Location: Scottsdale, Arizona, United States
Posts: 182
struct will be your friend here. If I have time and he doesn't reply before then I'll write you an unpack function tomorrow. Heading to bed otherwise I'd do it now.

e: I am pretty sure he gave wrong numbers for his example, and the distance should be 2B, not 3B, though I understand his stipulation that it should be if thinking logically.
Speed (KPH): 25
RPM: 4736
Temp (C): 87
Throttle %: 20
Lap: -1
Distance (KM): 5820

Last edited by vesrah; Apr 30th, 2015 at 11:50 am.
vesrah is offline  
post #5 of 5 (permalink) Old May 2nd, 2015, 11:31 am
Senior Member
 
Join Date: Aug 2006
Location: Sydney, NSW, Australia
Posts: 331
Yes there seems to be an error in my original example:

for the case of a "bike travelling at 25 km/h at 3200rpm with an engine temperature of 87 deg Celsius and 4284 km on the odometer at a steady 20% throttle opening would record the following hexadecimal values in the first 0.10 seconds:"

The initial bytes should read:
64 00 80 0C 7F 14 FF BC
10 00 80 0C 80 0C 14 80
0C 80 0C 64 00 80 0C 14

sorry about any confusion I might have caused. I did create ".CSV" export in my own application (for MacOS).

If you want an original copy of my White Paper or the MacOS application I created, let me know.

Andrew...
Andrew is offline  
Sponsored Links
Advertisement
 
Reply

Quick Reply
Message:
Options

Register Now



In order to be able to post messages on the Ducati.ms - The Ultimate Ducati Forum forums, you must first register.
Please enter your desired user name, your email address and other required details in the form below.

User Name:
Password
Please enter a password for your user account. Note that passwords are case-sensitive.

Password:


Confirm Password:
Email Address
Please enter a valid email address for yourself.

Email Address:
OR

Log-in









Human Verification

In order to verify that you are a human and not a spam bot, please enter the answer into the following box below based on the instructions contained in the graphic.



Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page
Display Modes
Linear Mode Linear Mode



Similar Threads
Thread Thread Starter Forum Replies Last Post
Import of a Ducati 900ss 1995 to USA Duc the German Ducati Motorcycle Chat 5 Jun 23rd, 2010 7:26 am
Import of a Ducati 900ss 1995 to USA Duc the German Supersport 6 Jun 23rd, 2010 7:22 am
Tempory Import Of Ducati From Uk To Usa Neil996R Hall of Wisdom 33 Feb 27th, 2008 8:28 am
Ducati Accessory Catalog in PDF format rummaggio Apparel and Accessory Chat 2 Jan 2nd, 2008 11:13 am
Analysis of Ducati 2005 Earnings Results Quantum11 Ducati Motorcycle Chat 52 Apr 9th, 2007 11:53 am

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On

 
For the best viewing experience please update your browser to Google Chrome