AI is a trend that has million opinions – positive and negative, so, as a technical person, I decided to get some personal experience on it. I am no expert in math and machine learning, so my point of view is from a user application developer. A Demo of my desired app is https://www.youtube.com/watch?v=ZV6M0O_VITE
To Open AI or not to Open AI
To collect some personal experience – I’ve been testing AI python code that uses open source language models for:
- text to speech https://github.com/snakers4/silero-models
- speech to text https://github.com/openai/whisper
- text prompt https://lmstudio.ai/
I’ve got a Open AI Wrapper https://programtom.com/dev/product/gpt-spring-boot-micro-service/.
But, for this pet project, I didn’t want to use OpenAI API to minimize fees and have my hands dirty in real terms. Otherwise – I would have been just REST API Integrator and will have not learned nothing from it.
Python Side
Everything needs to be splitted to the smallest building blocks – also called first principle in physics. In terms of artificial intelligence this is:
- Collecting data that will be used to train a machine learning model
- Transform the real world data to math models that will be used for future queries
- initialize the models, do the mapping and get desired result.
From all the above I’ve done only the third step, because I wanted to have app ready – faster.
Java/Kotlin Wrapper
I am not proficient in Python, but I am polyglot enough – to get my task done. I’ve made some flask endpoints of the above. I’ve loaded the language models from the examples in memory. This way it does not require loading the models anew. Finally – the simple flask app has parametrisized – the language to be grabbed from the request.
As a part of my desired app, I’ve also extended a previous project of mine. I’ve opened a REST endpoint that calls Google Translate API. It had some complexity as:
- it was implemented in Kotlin
- running on GraalVM
- Quarkus as base for
- It is part of Vaadin Web App – https://programtom.com/dev/product/poc-translations-manager-application/
In certain places – I’ve implemented query caching. No matter if one uses 3rd party API or personal one – caching will improve speed / use experience.
Flutter App
The end goal of my project is a Flutter App that
- records the voice https://pub.dev/packages/record
- transcribes it to text by sending the bytes with https://pub.dev/packages/retrofit file upload
- translates it to other languages (simple endpoint with passing source language, text and target language
- speeks out the text
- by passing the text to the backend and receiving the result it bytes
- cache it to file
- play it offline – on device
This may be useful for learning new languages or (somehow slow but – ) speeking to foreigners in their language. The app uses:
- Flutter Plugins – for iOS, Android & Windows for
- Recording https://pub.dev/packages/record
- Playing audio https://pub.dev/packages/audioplayers
- Widget Toolkit for the language picker: https://pub.dev/packages/widget_toolkit
- sqlite3, flutter_bloc, flow_builder, etc
I’ve got the app running on Windows and iOS, but I have some issues with Android. I also have several UX items to fix, but I hope I could manage to publish it to the stores soon.
Takeaways
I’ve experimented and learned a lot with this project.
- AI require a lot of and modern computing power (GPU) – even if you get already trained models. I have an old PC with upgraded RAM and SSD that runs Java Applications and Databases. But – for machine learning Applications – it is slow!
- While I learned a lot by experimenting different tech stacks and libraries, I’ve crossed an old point where complexity kills productivity. Keep it simple and stupid