Made a quick script so you can highlight words in the kindle UI which passes it through to stable diffusion and returns an image for the text that you have highlighted.

Not sure it is of any real use, and somewhat defeats the object of reading a book, but its pretty fun non-the-less.

At the moment it is just an addon for an unrelated project I was working on, but I can separate it off if people want a copy. Note that this still needs stable diffusion to be running on your pc - it isn't doing the generation on the kindle itself obviously

https://youtu.be/qK3bTm-ZN_Q

This is awesome--interested to hear more about how you are doing this. Please keep sharing as you develop this further.

It's not quite there yet, but the result I'm really hoping for is the ability to generate a map to represent the physical description of a scene. Often I find that really hard to visualize.

Thank you. I moved it onto one script for the pi, and one for pc. You can do it all on the kindle, but then you need to open stable diffusion up for your network, and I quite like getting the copy of the image in my pc folder too

You'd need to change the ip of your pc "stable_ip='192.168.0.2'" in "run_on_kindle.py", and change your kindle ip ("kindle_ip="192.168.0.213"") in run_on_pc.py

I also ran

iptables -A INPUT -p tcp --dport 5000 -i wlan0 -j ACCEPT
to open port 5000 on the kindle

and in stable diffusion I have the webui-user.bat set to:
set COMMANDLINE_ARGS= --xformers --api

to have the api running.

it basically just monitors
"/mnt/us/documents/My Clippings.txt"

for changes, and then passes that through as a request, then uses NiLuJe's fbink to set the image.

Btw this has no input verification, so use cautiously, but it should run ok. Next step would be to pull in the book name and author automatically and then set the "book_details=""" variable in run_on_pc, but I was just manually entering that for now

Maybe you could add support for https://www.mage.space/ , that way you wouldn't need a PI / computer anymore and just a wifi connection

Yea that would be cool to link to to an api, but it looks like theirs is closed and only for paying members:

"Do you have an API available?
Yes, we have a premium API available in limited beta. We are currently at capacity, but if you are interested, please contact us at mage@mage.space."

It could be done to link it to a google collab but you would still need to run that. I don't use any of the online ones as I run it locally, but it should be easy enough to add, but open to suggestions

I got it kind of working using stablehorde, but the image it downloads is a png in heic webp format. If I convert this using


Then it works on my pc, but it doesn't work on the kindle:
webp_image = Image.open("test2.png")
File "/mnt/us/python3/lib/python3.9/site-packages/PIL/Image.py", line 2994, in open
raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'test2.png'


I think it's due to the version of pillow, but I'm concerned that updating it will break the existing kindle stuff

-> I tried installing webp, but this went to "installing build dependencies, filled the kindle up and it rebooted. If I can get past this, or there's a way to get it as non webp from the api then this should work fine

Completely burnt out on trying to work out how to compile using toolchains.

I either need https://developers.google.com/speed/webp/docs/compiling this, a recent ffmpeg, or a recent pillow.

I have been attempting to compile webp using the kindle5 file here https://www.mobileread.com/forums/sh...d.php?t=342535 but trying to run it gives me

[root@kindle bin]# ./dwebp
./dwebp: line 1: syntax error: unexpected "("

which I understand to mean that it is not compiled correctly for the k4 (or 5, tested on both).

Is anyone able to compile it for me, or point me in the right direction for how I go about doing it? I have tested through wsl in windows, and collab

Thank you for any advice anyone can give, I'm really struggling to know where to go next.

That kind of unhelpful and weird error usually means you're trying to run a binary built for another arch.

(i.e., you did not, in fact, cross-compile, and have just fed an x86_64 binary to an armv7 machine ^^).

All of the things that you need are very very non-trivial, I strongly doubt that anyone is going to compile this for you as getting everything to work will be a huge time sink.

At a bare minimum, you'd need to cross compile all of the dependencies needed to build webp as mentioned by the page that you linked:
These probably have their own dependencies which again, you'd need to cross compile.

Once you're at that point, you might be able to cross compile webp. Looks like it uses Autotools(?) so you'd want to pass host + target options to ./configure to let it know that you want to build it for ARM.

TL;DR this is likely to be a huge pain and you'll probably have a less frustrating time if you figure out how to get that site to give you JPEG instead of WEBP or set up some kind of proxy to do this for you on a free AWS EC2 instance.

Cool thank you both for your advice

I went ahead last night and created an api that does it. Hopefully that is secure enough, but I'm just running that locally at the moment, but can run on a vps. Unfortunately their api just doesn't give the option to return a jpg/png. So frustratingly close heh, but the conversion using mine is quick enough.

I am going to make a video today when I can clear my desk, but here is my progress with it:
https://github.com/diggedypomme/Kindlefusion

I added a gallery for past saved images as well as the ability to manually search terms on stable diffusion and grab images from Unsplash. I have some pictures down at the bottom of the github


As for the compilation:

This is the copy from the colab that got a compile to work (but the outcome which wouldn't run) , but as you mention, I think I must have misunderstool the steps, and so this must be doing it in x64 as you say:

--------------


!wget https://storage.googleapis.com/downl...p-1.3.0.tar.gz

!tar -xvzf libwebp-1.3.0.tar.gz

!cd libwebp-1.3.0
!./configure --host=arm-kindle5-linux-gnueabi --prefix=/opt/arm-kindle5-linux-gnueabi_bcf9e45/sysroot/usr

!tar -xf arm-kindle5-linux-gnueabi_bcf9e45.tar.xz

!./configure --host=arm-kindle5-linux-gnueabi --prefix=$PWD/install

!cd libwebp-1.3.0
!/content/libwebp-1.3.0/configure --host=arm-kindle5-linux-gnueabi --prefix=$PWD/install

!make

!make install

!find / |grep dwebp

!zip -r /content/file.zip /content/install/

!ls /content/install/

from google.colab import files
files.download("/content/file.zip")

Oh and another feature that I'd like to try is to auto generate character pics for each person in a story, so that if you come back to it you can check their pictures. That would be very helpful for me. It could generate until you find one you like for each person and then auto load the same image when you highlight their name.

I had a play with a text generation tool textcortex.com :

"give me a 1 paragraph description of the physical features of the top 10 characters in the book Guards Guards by Terry Pratchett. These should be about 3 sentences long. make each description longer. even longer." (they were a bit short so I was just adding to it. A better prompt style could likely be used)

which gave:

Lord Vetinari: Tall, thin, and immaculately dressed with a calm, calculating gaze that belies his sharp intellect. He has a long, hawk-like nose and a receding hairline that adds to his air of authority. His finely tailored black suit with a high collar and a long, sweeping cloak gives him a commanding presence.

which gave:



I think that might be on to something, and it gives 10 free requests per day, so that should be enough to grab the descriptions for a book

Youtube video: https://youtu.be/SueGVpyrgG8

Short timelapse video: https://www.youtube.com/shorts/D7ejkuGo5Aw

I modified a picture frame and set it up to upload images as they are generated. I need to finish up the code to integrate the disparate parts, and add in the voice recognition (via a phone) then will be gifting them to friends