We have a large amount of Level 2 data from the Tokyo Stock exchange in a binary file format, and want to extract the level 1 quotes to a CSV file. The output file format is described in the first attachment below, while the second attachment contains the following documents required to parse the binary file:
E01_Market Information System FLEX Connection Specification Common Items DS [url removed, login to view]
E02_Market Information System FLEX Connection Specification Realtime Message (Standard) DS [url removed, login to view]
I've also uploaded a small sample file containing some data to be parsed. Note that the parser cannot keep all information in memory if more than one pass is required, as the daily data files are approximately 15 GB.
This project is preferably coded in C or C#, although Java or Python is also acceptable.
Note that the original Quote Format was missing the Issue Name of the symbol to which the quote applied, as I was thinking quotes for each name could be extracted to their own file. However it is probably better to aggregate all quotes in a single file, with the same name as the input file, but with "_Quotes.csv" added to the end of the filename.