Thursday, July 18, 2013

txtr Beagle - part 3 - storage and transfer protocol

I'm wrapping this up for now as one of the COG (chip-on-glass) devices has apparently fried and the reader has sold out.

UPDATE: Andreas Schier has written an open-source java toolchain for Beagle: https://github.com/schierla/jbeagle


UPDATE: Florian Echtler has built two Python scripts, one emulating the server and another one for the client. The server allows you to send images to your reader: http://floe.butterbrot.org/matrix/hacking/txtr/


Part 1: http://hackcorrelation.blogspot.com/2013/07/txtr-beagle-teardown-part-1.html

Part 2: http://hackcorrelation.blogspot.de/2013/07/txtr-beagle-part-two-software.html

Part 3: http://hackcorrelation.blogspot.de/2013/07/txtr-beagle-part-3-storage-and-transfer.html

Part 4: http://hackcorellation.blogspot.de/2013/07/txtr-beagle-card-parser.html

Part 5: http://hackcorrelation.blogspot.de/2013/07/txtr-beagle-native-code-analysis.html




I've scratched some of the white glue-like stuff away but the burn can be seen inside the glass. It was drawing abour 1A upon changing pages and the COG was getting very hot.


The current consumption seems to be at around 200mA when changing pages, 140mA when bluetooth is on and <10mA when running.


SD-card organization


The card is used as a raw continuous "file" split up into page sizes of 40000h (262144 bytes). The first page is just some setup and internal data (I assume) along with perhaps some indexes.
Starting with 40000h, each multiple of that contains one raw image: 40000h-7A980h, with the rest of the space containing 8 bytes of EOF(?) and blank data until the next mark (80000h).

The size 3A980h (240000 decimal) corresponds to a screen resolution of 600x800x4bpp, consistent to what the code was suggesting.
From each nibble only the upper 3 bits are used and the lower (LSB) one is left blank, not sure why. This means that there are 8 possible grayscale values.





I used IrfanViewer with a bit of the dataand the settings on the right. The result is on the left, at half width (IrfanView does not provide a 4bpp raw mode).

The second file requires a height of 800 confirming the correct data layout.
testfile1.zip
testfile2.zip


On my card the data stops at around the 16644610h mark, leading to a page count of 1565, consistent to the book loaded on it.

Transfer protocol


I haven't tested everything so I'm just describing what I could figure out by reverse-engineering.
The pages are rendered in grayscale on the Android device as described in the previous post. Prior to sending, a native method is called which converts the data into a bitmap of 600x800x4bpp and then compresses it using the standard DEFLATE algorithm.

The compressed byte[] array is then returned to the Dalvik code which sends it via bluetooth using the "PAGE " command. This particular command takes two parameters, the first one being the page number and the second one the byte[], one after another.
It's possible that the byte array is Base64-encoded, I haven't checked that.

In order to send a page, a new book must be started. This requires sending "BOOK ID="+number, "TITLE "+title, "AUTHOR "+author. After finalizing the transfer the "ENDBOOK" command is sent.

A good practice would be to check for all the acknowledgements and error codes sent back by the reader: BOOKOKTITLEOKAUTHOROKDELETEBOOKOKENDBOOKOK, PAGEOK.

I haven't touched upon the issue of pairing and partnership. This is done through commands like VIRGIN (remove partnership), GETPARTNER (returns NOPARTNER or a PARTNER_ID) and PARTNER_ID=(partners the phone with the reader through a generated 16 character hexadecimal string).


Post Scriptum


I would love if people that get their hands on them will start using the reader as a low-cost, low-power, large-area display alternative. The reader already provides plenty of 3.3V power, microSD slot and storage with breakout pads for the SPI and breakout for the BT module.
So the idea would be to turn the reader off, write custom data to the card, turn the reader back on. This was the main reason why I pulled this thing apart and documented the storage format.

Another idea would be to have another uC listen in on the BT connections while the reader is off (the cover screen can be replaced) and wake the reader up once-in-a-while to update the display and cover. The cover will only get updated while the device is powered on.


14 comments:

  1. Thank's very much. If you'd like some compensation for your broken reader I'd be willing to pay some (unfortunately they are sold out at the moment).

    ReplyDelete
  2. Thanks for the intentions :)
    It's ok, I'll just wait for some units to come in stock again. It's cheaper than buying those 4x20 LCD displays.
    The unit still works with the exception of the display, but I can figure out if it's changing pages or displaying something, BT works. Maybe some good will come out of it.

    ReplyDelete
  3. Modified the python scripts to use serial library for communication - works great :D

    https://dl.dropboxusercontent.com/u/1210045/txtr_hacked.png

    ReplyDelete
  4. @Ligius: thanks again for sharing, could you post a compressed SD card image somewhere (I wasn't able to take mine apart)?
    @mhmk: great to hear, could you send me a pull rq on github?

    Florian

    ReplyDelete
  5. @mhmk: could you provide your modified script ??

    ReplyDelete
  6. SD contents: https://www.dropbox.com/s/fopxhu1xk1zbtns/sdcard.gz

    In the near future I'll write a server script including PDF parsing in Java, since that's my language of choice. I'll ask some of you to help debugging until the readers are again in stock.

    ReplyDelete
  7. @oliver
    first version using serial (minimal modifications) http://bit.ly/14vvbaN
    second version with cleaned up code http://bit.ly/16TbENy

    of cause you need to set the correct serial port and adjust the filenames ;)

    ReplyDelete
  8. maybe I will find time next weekend to build a epub to image-pages renderer (in python)

    ReplyDelete
  9. @mhmk: I think you could try to build a Calibre plugin, then you wouldn't have to do the EPUB rendering yourself.

    ReplyDelete
  10. @floe, @mhmk
    I've also been working on the txtr beagle during last week. Thanks to floe's hint with the 2k window size for compression, I could finally finish rendering and upload to the device.
    I've already integrated all this into two calibre plugins (converter and device driver). I'll share it on github as soon as i've made some finishing touches later this week.

    ReplyDelete
    Replies
    1. @wassi any news on the calibre plugin?

      Delete
  11. @wassi: Awesome! Looking forward to testing your plugins.

    ReplyDelete
  12. I'll also update the information as soon as it becomes available.
    A new "episode" has been added, I consider it superfluous as Florian had written some information on the subject, but it might prove useful.

    ReplyDelete
  13. @wassi:

    How far did you get with the calibre plugin? I am really looking forward to having a more comfortable upload functionality for books. This plugin would be a blast!

    ReplyDelete