Thoughtworks - Technology Radar 34

Intro

Twice a year, Thoughtworks publishes its Technology Radar, an opinionated guide to the current technology landscape. It organizes notable technologies and practices (which they call blips) into four quadrants: Techniques, Tools, Platforms, and Languages & Frameworks; and four rings that express how confident they are in recommending each one, from "Adopt" down to what used to be called "Hold". I have been following the Radar for several years now, and reading each new edition has become something of a ritual for me. Volume 34 came out in April of 2026, but this time I was only able to read it in July, so parts of it already felt slightly outdated... which says as much about the current pace of the industry as it does about my reading backlog. Here are my thoughts on this edition.

General Comments

The first thing that caught my attention was a change to the Radar itself: the "Hold" ring has been renamed to "Caution". I dislike this change. It feels like an unnecessary cave in to the current hype machine, which insists that everything new must be flawless and that pointing out problems is bad for business. Being able to take a stand and argue against certain things, when one has strong arguments, is important for any serious conversation about technology, and softening the language of the ring that exists precisely for that purpose seems like a step in the wrong direction.

The distribution of blips across the rings tells an interesting story. There are 62 blips on Assess and 30 on Trial, against only 17 on Adopt and 9 on Caution. That overwhelming concentration in the middle rings feels very symptomatic of the times we are living in: there is an explosion of new things appearing, and it is necessary to at least keep track of most of them, while investing time to go deep into only a few. It is also striking how few blips landed on Caution - eight of the nine are Techniques, one is a Tool, and there are none at all in Platforms or Languages & Frameworks. The Platforms quadrant, in fact, has zero blips in both Adopt and Caution: everything in it sits in the middle, still being evaluated, with nothing yet considered either safe enough to fully recommend or problematic enough to warn against.

All four themes of this edition are related to AI: "The challenge of evaluating technology in an agentic world", "Retaining principles, relinquishing patterns", "Securing permission-hungry agents" and "Putting coding agents on a leash". It is the second time that this is the case, in previous editions there used to always be at least one theme dedicated to something else, as an effort to keep a broader view of the ecosystem. The four themes seem to form two couples: the first two are about how we humans are changing to adapt to this new model of working, while the last two are about the technical side of embedding generative AI models within a useful "exoskeleton" that is both effective and safe.

Highlights

OpenClaw is the only tool placed under Caution, and to me it is a clear portrait of the current state of AI hype. It made a huge impact when it was first released, and just a few weeks later it was already far less talked about... by now, it is rarely mentioned at all. I was always skeptical about the "hyper-personal AI assistant" category itself, as I see little value in delegating relevant choices of my life to an artificial intelligence. But beyond my personal preferences, the trajectory of OpenClaw illustrates the broader pattern: there is a huge push towards adopting every new thing that appears, without any clear success story that has proved itself over sufficient circumstances.

The agentic coding tools, on the other hand, are where the consolidation is most visible. Cursor reached Adopt, and it was one of the first agentic IDEs I tried, early on. I had an overall good experience with it, though not a very remarkable one. It holds the distinction of being the only agentic coding tool with which I hit a token budget limit while coding (the only other time that happened was with Claude Code, and that was because I was testing Fable, a model with a more restrictive budget). Since the limit appeared unexpectedly, midway through development, I switched to Windsurf and never went back: overall, I preferred working with Windsurf. On Assess, Pi is one I find very promising: I have seen some very interesting proof of concept projects using the Pi SDK as a framework on top of which entire TypeScript systems are built, UI and backend included. It seems like a great candidate to bridge the gap between AI agents and traditional applications, enabling truly agentic applications, and I am very interested in testing it soon. Also on Assess is OpenCode, which became a true milestone in my routine - it is now my tool of choice for simpler or more exploratory tasks, and I wrote a dedicated post about it recently. And finally, Claude Code reached Adopt, having indeed become the industry standard for agentic AI coding: I have been using it extensively, reserving it for the complex tasks that demand state-of-the-art intelligence.

Small language models appearing on Assess caught my attention for a different reason. By "small", the Radar means models of 3B parameters and below, which sits just outside the range I most commonly explore in my personal open source projects - JenAI, LLP and Chargen are built to work with models in the 8B to 70B range. Adding the frontier models I use for development, there is now a whole spectrum of model sizes, each serving its own purpose. I like to think this shows we are starting to reach some level of maturity in employing AI for actual use cases, moving beyond mere experimentation: choosing the right size of model for the job is the kind of engineering decision that only becomes relevant once the technology is being put to real work.

Curated shared instructions for software teams reaching Adopt was particularly satisfying to see. As an InnerSource champion, I have been talking within my company about how AGENTS.md and similar instruction guidelines for agents are now an important and integral part of any project, and how they enable more effective collaboration even from outside contributors. I have given a series of talks about this in internal events already, and I plan to at least write a blog post on the topic soon.

Codebase cognitive debt, placed under Caution, names one of my main concerns with the current shift to AI-assisted software engineering. I have developed software across the whole spectrum that goes from full vibe-coding (just ask the AI and go with whatever it produces, without even looking at the code) to purely human-written code, and I have had successful results at every point of it. But the characteristics of those successes are different, and being distant from the code still does not feel as reliable and sustainable as being hands-on. A big part of being proficient at developing a software system lies in the mental map one has of it, and not being actively engaged in writing the code erodes that map very quickly. This fundamentally changes the process of software engineering. When humans write all of the code, the more you work on a system, the better you become at working on it, because your mental map keeps expanding. When humans stop looking at the code, the relationship inverts: the more you work on the system, the less proficient you become, as the code evolves in ways you can no longer clearly picture. There is no turning back, and the current push in the industry is for humans to be as far out of the loop as possible in order to gain development speed. But we have not lived with this model long enough to have clear success stories that have stood the test of time and proved it sustainable... and until we have those stories, I remain cautiously skeptical.

Things Tested And To Test

Beyond the highlights, several other blips crossed paths with my own explorations. Among the things I have already tested are LangGraph (which I have used as my default choice of AI agents framework - the Radar moved it out of Adopt and into Trial, as there are newer alternatives that can compete with it now), Google's Agent Development Kit (ADK) (which I used briefly when assessing different agent frameworks, and enjoyed more than other alternatives I also tried such as CrewAI, as it maps better to a software engineering mindset - the Radar's main criticism is that it is still pre-GA) and HTML Tools (the first time I see this explicitly named, but an approach I have been following for a while: whenever I ask a coding agent for a report, I ask for the findings consolidated in a nicely stylized standalone HTML page, which reads much better than the default markdown).

On the list of things I want to test are Mastra (a TypeScript-native open source framework for building AI applications and agents), team of coding agents (a technique I have only limited experience with so far), code intelligence as agentic tooling (I have not yet explored much of the Language Server Protocol as a way for LLMs to work more efficiently, but it is high on my list) and mutation testing (an idea I have known for years and was always skeptical about for human-written code, but as a "test-the-test" approach for AI-generated test suites, which realistically no one will fully review, it finally starts making sense to me).

Closing Thoughts

Stepping back, the picture this edition paints matches what I see in my daily work. Kent Beck describes the life of a product in three phases, his "3X" model: Explore, when you cheaply try many things in search of what works; Expand, when something has caught on and you scale it while removing bottlenecks; and Extract, when the ground is stable enough to optimize for efficiency. The AI-dominated landscape of this Radar feels like an industry moving from Explore into Expand: the explosion of new things being created continues (hence the crowded Assess and Trial rings), consolidation is beginning around a few winners (hence Claude Code and Cursor on Adopt), but there is still no concrete success story that has stood the test of time... which is precisely why so much of the Radar remains in the middle rings, and why I keep some healthy skepticism about the most aggressive promises being made.

The next edition of the Technology Radar should be released around October or November of this year. I intend to once again read it (hopefully closer to the release date this time) and share my thoughts here on the blog.

Exploring opencode

I started using opencode around May of 2026, and it truly felt like a milestone in incorporating AI into my daily routine. I mentioned in my May monthly recap that I intended to write a dedicated post about the experience, this is that post.

Some context first. I started using AI coding agents (or harnesses, as I prefer to call them) around August of 2025, with Cline on VSCode, and then moved on to Claude Code in January of 2026. In both cases this happened at work, for purely professional purposes, coding and other software engineering tasks. It was only when I began experimenting with these tools on my personal computer, for a much broader range of tasks, that I eventually discovered opencode. It was a spark that changed a lot of how I think about and use harnesses, for a few reasons.

What Sets It Apart

Install And Start Using

The first thing that struck me is how simple the setup is. With every other harness I have used or looked into (Cline, Claude Code, Codex, etc.), there is a whole ritual before you can do anything: create an account, choose a subscription plan, log in through the tool, and so on. With opencode, you install it and start using it. That is the entire onboarding. It sounds like a small thing, but it completely changes the feeling of picking up the tool for the first time.

Free Access To Reasonably Good Models

The second reason is that opencode currently offers free access to reasonably good models, without any severe limits. The obvious caveat: it is impossible to know for how long this will be sustainable, and opencode might well start limiting usage or requiring a paid subscription in the future. The slightly less obvious caveat: every session needs to be treated as likely shared with the companies that created the model, as the simplest benefit a company can get from giving access to their models for free is data for future training... so, best not to share any personal or sensitive information. Regardless, right now it makes opencode a perfect tool for experimentation.

By lowering the cost of exploration to basically zero, it allows everyone to try the harness for all sorts of things, not just coding. These tools have access to terminal commands, beyond reading files and accessing the web - which means that anything that can be done on a computer can be explored through AI with them. And a huge amount of our society runs on computers, so the potential is enormous. To give a few examples of things I have done: searching for a public, free web radio station from any country (Argentine rock forever!) and playing it directly through the terminal, looking up lyrics and other info about new music I found, and debugging and fixing a broken game installation by letting the agent look at the actual configuration files. Without the burden of knowing you have a certain limit of tokens to spend, you feel free to explore all sorts of random things that an AI agent might do with a computer.

I am even considering making opencode the very first thing I install, the next time I setup a brand new OS installation (likely Linux, as it is my default OS). I find it intriguing to think what an AI-based distro would look like, though I would not want to go to the extreme yet. In my mind just being able to bootstrap the OS with opencode, and then using it to install everything else that I need (as it would have access to the package manager of the distro) by just talking with the computer feels really cool.

Open And Provider-Agnostic

The third reason is that opencode is a fully detached, open source agentic harness that can work with models from several providers. Most tools in this category fail at least one of these criteria: they are IDE-first (Cline), closed source, or tied to the provider that created them (Claude Code with Anthropic, Codex with OpenAI). Pi is an alternative that is also detached, open source and provider-agnostic, but it is far less complete when it comes to features and capabilities.

Being provider-agnostic also means opencode can be used with local models. So even if the free access to hosted models disappears in the future, it will still be possible to run it for free with local ones. That is a kind of insurance that no closed, single-provider tool can offer.

Limitations

Despite all the positive aspects, there are some limitations as well:

Model Quality

While the free hosted models are good, they do not reach the same level of quality as frontier models. For more complex or nuanced tasks, they often do not achieve the best results. This can be worked around with more guidance and handholding from the user, but it takes more work. For the absolute best quality, proprietary models (which usually work better with their own company's harness) are still the way to go.

Context Management

Context management is often a problem. Most of the free models offered have a context size of at most 128k tokens, which gets consumed very quickly - the tool seems to be quite aggressive about how much it puts into the context. It does perform automatic summarization when it gets close to filling the available context, which is good, but more often than not the summarization loses some information that is essential to the task at hand. In practice, this means the quality of the results degrades very quickly, making opencode not very suitable for long-running tasks.

Closing Thoughts

For me, the real significance of opencode is not any single feature, but what the combination enables: a tool that anyone can install and immediately start using to explore what AI agents can do with a computer, at no cost. The frontier of quality still belongs to the proprietary models and their harnesses, and the limitations around context are real. But as a gateway for experimentation, and as an open, provider-agnostic foundation that will keep working regardless of what any single company decides, it has earned a permanent place in my toolbox. I expect to keep exploring what these agents can do well beyond coding, and I am sure some of those experiments will end up here on the blog.

Project: LicLacMoe

Description

LicLacMoe is a desktop application that allows you to play tic-tac-toe against local Large Language Models (LLMs).

I released the first official version in May 4th, 2025. The name, of course, is a word play with tic-tac-toe, replacing each first letter with the initials of "large language model".

The application assumes you have an LLM server running on your machine (this is a deliberate choice), by default on port 8080 but that is configurable. It presents the player with a visual tic-tac-toe grid that can be used for playing - once the player makes their move, a call to the local LLM server is made with the current state of the match, so that the LLM can pick the next move of the AI opponent. The entire interaction with the LLM is through playing, with no conversational interface.

I developed LicLacMoe as a way to explore using LLMs in a way that does not involve any conversation between the user and the AI system. Chatbots have become almost synonymous with LLMs, in large part due to how they were popularized, so it was an interesting experiment to use them in a completely different manner.

Context

I developed LicLacMoe as I was exploring how to build systems using generative AI technologies for the first time. I blogged about the set of 4 projects that came out of this exercise in my post entitled "First Steps Into AI Engineering". LicLacMoe was the fourth and last of these, and the most purely exploratory one.

As described in the post just mentioned, for LicLacMoe I wanted to write most of the code myself, without relying too much on frameworks, so as to get a better feeling about working with these models. It is also intentional to explore only open and local AI models, as it is my intention to find out how far can we go working only with models that can be fully personal and owned by its users.

Highlights

Interacting with LLMs without chatting

LLMs have caught our full attention due to their uncanny ability to behave like a human being in a conversation. However, the big question on everyone's minds was if the models actually have some degree of reasoning intelligence, or if they are just really good at reproducing our patterns of communication (of course, it must be mentioned the obvious philosophical question: "could it be that there is no difference?"). To a certain degree, the appearance of reasoning models, and the current trend of agentic AI, have shown that LLMs can definitely be exploited for some amount of reasoning intelligence, but at large scales the question still remains. My first intent with creating an application that uses LLMs without any chat interface was to see how it would feel to use LLMs purely as a source of thinking, without any verbal communication. While the long time it takes for it to generate an answer can be a bit frustrating, overall it was a positive experience - it is a really, really weird way of interacting with a computer system.

My second intent was to just get used to incorporating LLM answers in a bigger system, as part of the User Interface. I honestly think chatting (especially when you have to type long messages) is not the best interface for any complex computer system, far from that. In order to make full use of the potential that generative AI systems have, we must learn to incorporate them seamlessly into our flows, and that includes into our computerized applications. This was just a first step in this direction, I have several other ideas I want to explore further with regards to this.

Not needing to code game rules and strategy

As mentioned previously, using LLMs for pure intelligence is a really weird experience. One of the weirdest parts was that, in order to implement LicLacMoe, I did not have to implement a strategy that knew the rules of tic-tac-toe at all. I still implemented the logic of the game in order to verify the end result of matches, but I think with a little more development time I could have even replaced that with well-crafted prompts.

I am sure that this was in big part due to tic-tac-toe being an extremely simple and popular game. It is reasonable to assume that most (if not all) models will have seen enough examples of matches and descriptions of the game to be have memorized a pretty good understanding of how to play it. The same would most likely not be the case for more complex games - I find it very interesting to think about how complex of a game is it possible to teach LLMs simply by feeding it enough cases.

Regardless, it felt very odd to rely on a system that "just knew" the rules, and to which I could just feed the current state of the match and it would produce a next move. Of course, it would not always be a valid move (error handling and retry policies were more essential in here than in any other LLM-based system I have implemented so far), nor a particularly brilliant one. But even small models would consistently give something workable in a reasonable amount of time (and retries).

Reasoning vs non-reasoning models

This leads into the final interesting note. While testing the application, I found that non-reasoning models would mostly generate moves that looked a bit random, and could very easily be defeated. I had to make some changes to the logic of parsing the answer from the LLM to support using reasoning models - however, changing to these models drastically improved the performance of the AI player. I tested it with Qwen 3, 8B parameters, 8 bit quantization - a rather small model as far as LLMs go. In comparison, the non-reasoning model I used was Gemma 3, 27B parameters, 8 bit quantization, a model more than 3 times the size. While I have never been a huge fan of reasoning models (to my common use cases they usually don't offer too much improvement, and are considerably slower), in this particular case it was easy to see the value that such models bring.

Future Expansions

Benchmark of performances

As mentioned before, while testing the application I used a non-reasoning 27B parameters models (which had bad performance) and an 8B reasoning model (with significantly better performance). One thing I would like to do, if I ever have the time to, is make a more comprehensive list of the performance for several models of different families and sizes. I would be especially interested in seeing how small in size we could go with a reasoning model and still have it able to avoid defeat in most matches. I would be pleasantly surprised if this is possible with a model smaller than 4B.

Induce reasoning for non-reasoning models

Another interesting exploration would be to craft the base prompt so that even non-reasoning models would think about the current state first before choosing a move. This is easily done with very popular techniques to force step-by-step thinking. It would involve changing the base prompt and possibly the parsing of the response as well. This could then be compared with the improvement in performance gained when switching to a reasoning model, to see if the training that these models receive in reasoning actually gives them an advantage or not.

Model vs model

Finally, the last extension I might make is to change the game to support AI vs AI mode, with two LLMs playing against each other. This could then allow tournaments to be played, and metrics to be gathered as to which models performs better against each other. It would be a nice and fun addition, but it probably won't be a priority any time soon for me.

Setup

Although LicLacMoe is not one of my portfolio projects (which have a fixed set of quality standards I expect to maintain through their entire lifecycle), I did configure most of the foundations I use for those.

I use Github Actions to generate a new release for LicLacMoe whenever new code is pushed into the main branch and alters relevant core files of the project.

I have a changelog file, a file with guidelines about contributing and an architecture.md file (an idea I adapted from this great article), beyond the usual readme file as documentation.

As this project was done in an exploratory and proof-of-concept approach, I did not include automated tests. This is the main point of departure with regards to the quality standards I expect from my portfolio projects.

Links

Source code: Github

Executable: Releases

Project: LLP (Local Language Practice)

Description

Local Language Practice (LLP) is a desktop application to practice languages through a chat roleplay with local Large Language Models (LLMs).

I wrote a sketch version of it around May of 2024, and released an official version in April 16th, 2025.

The application assumes you have an LLM server running on your machine (this is a deliberate choice), by default on port 8080 but that is configurable. There are two main features in the application, both present in the main view: a chat between two characters (the human user acting as one, with the LLM acting as the other) in a particular language and widgets for translating to-and-from between that language and English. The default scene is a conversation between two robots in a futuristic world, but custom scenes can also be imported to be played out. As of the first released version, the supported languages are French, German, Portuguese and Spanish.

I developed LLP as a tool for personal usage, following closely my particular flow for practicing languages. Although it is not intended for use by a general audience, I expected it to be functional and useful from the start, and not merely a proof of concept.

Context

I developed LLP as I was exploring how to build systems using generative AI technologies for the first time. I blogged about the set of 4 projects that came out of this exercise in my post entitled "First Steps Into AI Engineering". LLP was the third of these 4, and the first to work with more than one chat context.

As described in the post just mentioned, for LLP I wanted to write most of the code myself, without relying too much on frameworks, so as to get a better feeling about working with these models. It is also intentional to explore only open and local AI models, as it is my intention to find out how far can we go working only with models that can be fully personal and owned by its users.

Highlights

LLP was the first of the First Steps Into AI Engineering projects that I originally intended as an useful application. While JenAI eventually became my main way of interacting with LLMs, at first it was going to be just a proof of concept to learn how to handle conversation state - LLP, on the contrary, was planned from the start to automate one of my personal flows for learning languages. This made the entire development process very satisfying, as with each new part implemented I was able to really see the benefits it added. It also made working with user stories and a project board to track development very intuitive and fruitful. While previously I have used LLMs to generate user stories (SnakeJS in particular), this time I found it more efficient to just write them myself, as I had a very good idea of everything I wanted in the application and how to communicate that.

The flow I wanted to automate with the application is the one I use to bridge the gap between self-contained lessons and real-world usage, when learning languages. While I appreciate and enjoy doing lessons, both in applications such as Duolingo and in more traditional textbook format, I find that by themselves they do not really prepare one to actually use the language being studied in real scenarios. So I tend to complement them with other forms of practicing the language, such as listening to music with lyrics in that language, or reading books written in it. More recently, with LLMs becoming really powerful and popular, I started using the frontier models (as they tend to be much more reliable for multilanguage usage) to simulate conversation in the languages I study, as a way to get a more immersive practice - it has proved to be quite effective. While the conversation is the biggest part of this practice, I find it useful to also have separate tabs open with translation tools, in which I can paste any part of the conversation which I do not feel confident about (either in understanding or in creating) and immediately get a translation. I designed LLP so that I could have both these things in the same view, so that I would never need to leave it for the entire practice session.

A delightful challenge in this project was crafting the prompts to get the LLM to behave exactly as expected for each process. While the conversation one was the closest to what I had done in the past, especially with JenAI, it also had the nuance of needing to stay in the specific language being practiced, and of staying coherent with the AI character while also understanding it should act as a tool for language practice. It took some iterations to get the conversation prompts right, but a combination of a system prompt setting up the scene and the task, and a new system prompt (injected and removed from the conversation history as needed) right before each response did the trick reasonably well. The prompt for the translation process had to be more specific and less open ended than the conversation one, and I was able to leverage my learnings from working on Chargen when crafting it. One of the tricks I have been finding very useful to get LLM models to generate outputs in a specific format, even when they were not specifically trained for that, is to just tell them in the prompt that their response will be piped directly to an automated system that will process it, without any human being involved - this alone seems to prevent most models from adding unnecessary explanations or deviating from the specified format, in my experience.

I am very fond of the feature for loading custom scenes, as it provides a way to explore unique settings without needing to change the source code and generating a new release. Since I consider myself the only audience for the application for now, I am happy to keep it based on manually customizing JSON files, instead of creating a separate custom creator app or feature inside LLP for this - it might change in the future, if I decide to make a more widely appealing version of the application.

Future Expansions

Saving session state

The main feature I am considering to add is the ability of saving practice sessions with the conversation history to be loaded later. The application already has the capability to load custom scenes, but they always start the conversation from scratch. Being able to save a conversation to continue at a later time should be very useful (as I have already seen with JenAI). I am not sure when I will have more time to work on this, though.

Using industry standard request framework

Another change I would like to do is replacing the custom logic to interact with the LLM server, which I have been reusing and slightly tweaking and improving between each project of this series, with a more industry standard one. While the classes I have been using have been great for me as a learning tool, and have not so far caused any major issues, for LLP in specific they are problematic: they constantly break when using any slightly more complex character (such as accented versions of vowels), which led me to create helper sanitizing functions to just remove any problematic character and replace it with a safer version whenever possible. For English-only conversations, this approach works - but for practicing other languages it messes up the specifics of the language quite bad. Most non-English languages have a heavy usage of non-ASCII characters, and learning them with an application that just ignores these characters is obviously a bad idea. I am looking for alternatives, and in special for Spring AI as a good replacement to enhance this.

Adding new languages

Adding new languages is also something I will quite probably do in the future, mostly whenever I decide to pick up another language to learn. I have decided to hard code the supported languages in the application, and I do not intend to change this in the future. Given that I consider myself the only audience for the app, coding a new one whenever I need to is a much smaller hassle than making the list dynamic in the first place.

Generally usable version with another framework

Of all projects in the First Steps Into AI Engineering series, I see LLP as the most potentially useful for a large audience. However, I do not think that an application built on top of Swing is viable for a modern public, in 2025. So, one of the things I might do in the future (although I am not currently seeing this as a priority) is to create a new application with the same features as LLP, but coded in a more modern framework (probably with a different, more frontend-friendly language as well). This would give me an opportunity to recreate several parts of the application in a more abstract and dynamic manner.

Custom scene creator

Finally, the last expansion that I imagine for LLP would be a system (either in-app or as a separate project) to create custom scenes. Currently, this depends on manually editing JSON files - which is perfectly fine for me, but would be irritating for anyone else. If I decide to create a version for general use, this will definitely be in scope, but for now I do not have plans to implement it.

Setup

LLP was the first of the First Steps Into AI Engineering projects that I wrote from the start with the intention of using as a portfolio project. Although I started this series as a space for exploration and learning, over time the projects acquired a level of scope and complexity that made me elevate them into more serious projects (although still not intended for a general audience). Due to this, I configured all of the foundations I use for portfolio projects.

I use Github Actions to generate a new release for LLP whenever new code is pushed into the main branch and alters relevant core files of the project.

I have a changelog file, a file with guidelines about contributing and an architecture.md file (an idea I adapted from this great article), beyond the usual readme file as documentation.

I also included automated tests for everything except the UI code. As of version 1.1.0, all non-UI packages had more than 75% method test coverage. I made extensive use of Test-Driven Development (TDD) for the development of the project.

Links

Source code: Github

Executable: Releases

Event: The Developer's Conference 2024-6 Porto Alegre (onsite)

The Developer's Conference (TDC) is arguably the most important technology event for software developers in Brazil. It is regularly held around four to six times a year, each time in one of the recurring host cities (São Paulo, Florianópolis and Porto Alegre being some of the most common ones). The talks presented in each edition span a wide variety of topics, giving a great overview of the technology ecosystem as a whole. 

Since 2020, most editions have also been transmitted online, either in a fully remote format or in a hybrid one. I have written in a past blog post about how I have been a regular attendant ever since my first participation in 2019.

Porto Alegre edition for 2024

While most of the times I attend remotely, whenever there is an edition in Porto Alegre I try to be onsite for the event. This was the case for the final edition of TDC in 2024, which took place in Porto Alegre between December 4th and December 5th. 

Going back to the familiar Uniritter campus and meeting in person with some of the most important figures of the national software development scene, as well as colleagues from the past whom I had not seen in years, is always an amazing experience!

Summit AI format

The Porto Alegre edition for this year was in the new "Summit AI" format, in which there are only two days of event and three main stages (plus several other venues such as stands and digital rooms), with all of the talks focusing on the topic of AI. Overall, there were 4 editions in this format this year, with 2 other (in São Paulo and Florianópolis) using the more traditional tracks format. 

While I understand the need for a more frequent and deeper discussion about AI, given the overwhelming speed with which it took hold of the conversation in the tech industry since ChatGPT's launch on November 2022, I admit that I prefer the traditional format, with more tracks and thus more talks and a wider variety of topics being discussed.

Mute talk format

One feature that was new to me, although I believe it was already used in previous editions this year, was the use of multiple talks happening in the same place at the same time, with headsets given to the attendants that would allow them to choose which talk they wanted to listen to. 

While it allowed to easily switch talks whenever there was a dull moment in one (very reminiscent of the online experience of switching rooms in the streaming platform with a single click), I did not find this novelty to be particularly helpful, from an attendant's point of view (although I'm sure it made the logistics much simpler). I also thought that the different talks interfered slightly with one another, as a loud ovation would sometimes interrupt a speaker mid-sentence when one talk ended while the other was still in progress.

Sessions

These were the talks I attended:

Day 1: 2024-12-04

• Nova Indústria Brasil: as ações da Finep para apoiar o desenvolvimento de IA no país. (New Industry Brazil: Finep's Actions To Help The Development Of AI In The Country)
• Inovação no Varejo: A Nova Era das Lojas Automatizadas com IA (Innovation In Retail: The New Age of Shops Automated With AI)
• Inteligência Artificial a Serviço do Farmacêutico: Consulta Interativa de Bulas de Medicamentos (Artificial Intelligence At The Service Of The Pharmacist: Interactive Consultation Of Medicines Leaflets)
• Reimagine o desenvolvimento de software com IA generativa (Reimagine Software Development With Generative AI)
• LLMonsters: desafios de GenAI em produção (LLMonsters: Challenges Of GenAI In Production)
• Discovery Kanbanizado - O que garantir antes do compromisso (Kanban-ized Discovery: What To Guarantee Before Committing)

Day 2: 2024-12-05

• Potencializando a Agilidade com Inteligência Artificial: Transformando Decisões e Processos em Tempo Real (Powering Agility With Artificial Intelligence: Transforming Decisions And Processes in Real Time)
• Uso de Agentes baseados em LLMs como suporte a equipes (Usage Of LLM-Based Agents As Support To Teams)
• Aplique AI e deixe agir: como implementar agentes de IA generativa (Apply AI And Let Act: How To Implement Generative AI Agents)
• Engenheiro de Prompt: A Arte de Fazer Perguntas que Movem a Era da Inteligência Artificial (Prompt Engineer: The Art Of Asking Questions That Move The Age Of Artificial Intelligence)

Takeaways

Attending this edition was a very good learning experience. I was able to hear about the implementation of LLMs for real use cases and in production scenarios - as someone who has been following closely this area, and who has been implementing several personal projects using LLMs, this was extremely insightful. I was also able to get a good overview of agentic AI, a subarea that I hope to dive deeper into in the near future but that I have not yet explored much. 

Finally, I was able to meet a few people whom I had not seen in a long time, which is always nice and heartwarming. As always, I intend to continue attending every edition of TDC that I can, and I hope to next year see more editions happening in the traditional tracks format instead of the Summit AI one.

Project: Chargen

Description

Chargen is a desktop tool written in Java to generate images and biographies for fantasy characters based on their skills.

I wrote a sketch version of it around May of 2024, and released an official version in November 29th, 2024.

The application assumes you have an LLM server running on your machine (this is a deliberate choice), on port 8080 (I plan to make this configurable in the near future), and a Stable Diffusion server for image generation also running on your machine on port 7860 (should also be configurable in the near future). The application presents a graphical interface for entering the information about the character, such as their name, class and skills, and two buttons: one to generate an avatar image for the character, and the other to generate a textual biography. Both use the information entered about the character to generate appropriate outputs. The biography and avatar generated can be saved to disk as text and image.

Chargen was developed in the spirit of a proof-of-concept, so that I could explore interacting with two different generative AI tools in the same application and learn from this experience. It is not intended to be used by a general audience.

Context

I developed Chargen as I was exploring how to build systems using generative AI technologies for the first time. I blogged about the set of 4 projects that came out of this exercise in my post entitled "First Steps Into AI Engineering". Chargen was the second of these 4, and the first one to deal with image generation.

As described in the post just mentioned, for Chargen I wanted to write most of the code myself, without relying too much on frameworks, so as to get a better feeling about working with these models. It is also intentional to explore only open and local AI models, as it is my intention to find out how far can we go working only with models that can be fully personal and owned by its users.

For Chargen I chose to work with text generation and image generation as separate tools instead of using a LLM that also has image output, as those are not yet very widespread as open source models.

Highlights

The biggest benefit of this project to me was learning how to generate images programmatically with Stable Diffusion. Although I had already used it quite a bit, it had always been through some provided UI, such as the classic text-generation-webui by Oobabooga. Getting used to access this service through code opened up my mind to a lot of cool ideas that hopefully I will be able to explore in the future.

Another good point was getting familiar with using LLMs as a single-purpose tool. I have always liked Simon Willison's analogy of seeing LLMs as a "calculator for words", and I think that one of the reasons why this requires some mental effort to achieve is that the usual way we interact with LLMs - as a chatbot - steers our thinking towards the opposite direction. We get lost in the simulation of emotions, the rhetoric and the stealthy handling of conversation state, and that makes it hard to see the system we are interacting with as something that is just processing some text to generate other text. Using them in the context of Chargen, in which every request has a very specific type of output and fulfills one very specific need, without any context or state from previous messages, helps the vision of LLMs being a calculator.

Chargen also gave me an opportunity to work with prompts in a more nuanced way. When developing a simple chatbot, usually you only think of a good system prompt to guide the tone of the conversation, and from that point onwards just relay to the model the state of the conversation plus the new message from the human interacting with it. In Chargen, however, both prompts (for generating the avatar and the biography) were tailored to produce a very specific result, almost as a function. Thinking about how to integrate the user input (in the form of the character's attributes) with the function-like prompt was an enlightening experience.

One thing that ended up being challenging was coming up with a prompt to generate good avatars that did not fall into the uncanny valley. I believe this was mostly because I chose to restrict myself to working with the base Stable Diffusion 1.5 model, which is quite old by now and nowhere near being among the best ones you can find even in the open source landscape. The fact that I did not want to include any trademarketed words in the prompt also had a big impact. I could not find a prompt that would generate excellent images, so I settled for one that would generate reasonably acceptable ones, most of the time. For any application looking for more robust results, this should be easily fixable by changing the two major points I mentioned.

For this project I wanted to work only with Java and focus on learning about interacting with generative AI models, so I picked a GUI library that does not generate the most beautiful applications ever. Swing had the advantage that it requires no external dependencies, that it is fairly straightforward to use and that I had already used it in several other projects in the past. But neither the code to create the views nor the views themselves end up particularly beautiful, so for any serious attempt at creating a similar application for a wider audience, a better GUI engine should be chosen. I do not intend to spend too much time improving the UI either.

Chargen requires two generative AI models to be running on your local machine at the same time. That makes it almost impossible to expect anyone else to run it by themselves to check out, so I am not investing into making it too user-friendly. Instead, I see it as a successful proof-of-concept, that gives good indications on how to build a similar system for widespread use if desired - one that should definitely then consume remotely hosted models. I was a little surprised to discover that my machine handled pretty well running all the required setup to make Chargen work - but then again, I am a software developer interested in AI that also likes to play games on his PC, so my machine is not really an average end-user's one.

Future Expansions

As mentioned previously, I do not intend to spend too much time improving Chargen, as I already reaped most of the benefits I expected from it in the initial development. It could certainly benefit from some work to make the UI more pleasant and a deeper research for image generation models that could put out better results, but I am content with the current state as I do not expect it to be used by other people.

I do intend to make it slightly more customizable, though. Especially by allowing for a similar configuration through command-line options as JenAI has. The most important parameters being the ports for the LLM and the Stable Diffusion models. I will have to make the "model" property configurable in order to support ollama as a backend as well. I might make the prompts configurable in the future, but do not plan to do so as of now.

Another possible expansion would be to add more constraints to the character attributes, such as a maximum number of total points to be distributed, or drop-down options for details that can be restricted to a set. If I do implement this in the future, I would prefer to make it an optional feature instead of replacing the current, freer, functionality. I do not see a very strong reason to restrict the possibilities that the generative AI models give to a creative user in this context.

Setup

Although Chargen is not one of my portfolio projects (which have a fixed set of quality standards I expect to maintain through their entire lifecycle), I did configure most of the foundations I use for those.

I use Github Actions to generate a new release for Chargen whenever new code is pushed into the main branch and alters relevant core files of the project.

I have a changelog file, a file with guidelines about contributing and an architecture.md file (an idea I adapted from this great article), beyond the usual readme file as documentation.

As this project was done in an exploratory and proof-of-concept approach, I did not include automated tests. This is the main point of departure with regards to the quality standards I expect from my portfolio projects.

Links

Source code: Github

Executable: Releases

Review - CoIntelligence: Living And Working With AI

Since the launch of ChatGPT, on November 2022, Generative AI has been the hottest topic in every conversation. It is almost two years later now, and we continue seeing new breakthroughs happening every other week. Trying to keep up with every news and opinion piece in such a context is impossible, but at the same time it is essential to be aware of the implications that this technology will have in our future. In cases like this, the best solution is to turn to the prominent voices, those who have been out there trying and thinking deeply about this topic for a long while, and being recognized as having valuable insights to offer.

Given that, I picked up the book CoIntelligence: Living And Working With AI, by Ethan Mollick, to hopefully serve as a baseline of understanding when it comes to the current trend of Artificial Intelligence. Ethan is a college professor at the University of Pennsylvania who has been experimenting and writing profusely about AI (following him on Twitter/X is among the best things you can do to keep yourself up to date with the practical side of the field) and, if you believe his cleverly AI-directed propaganda on his bio page, is a well-respected person by artificial intelligences of all kinds - who better to bring some insight to us, mere biological, natural-intelligence plebs?

I read CoIntelligence in the context of my personal studies habit, in which I set off 15-30 minutes of my free time each day to work through a technical book which is either a classic in the field or that I believe has an interesting point of view. I studied this book in 2024, from April 22nd to June 8th. In this post I offer a short review of the book, consolidating some of my notes and highlighting the things that drew my attention the most. If you are interested in the field of AI, I strongly recommend you buy the book and read it for yourself - it is worth every second you spend with it.

Motivation

I have been interested in the field of AI for as long as I remember, long before I decided to have a career in Technology. Even as a child, I was draw to videogames and wondered how might a piece of electronics have enough intelligence to put up a challenge against human players - I vividly remember mixing up the concepts of soul, intelligence and conscience, wondering if after death I could be re-incarnated as a NPC in some game. Later, when I became a Computer Science undergraduate, I chose to follow AI as my main field of interest, committed to attending as many classes in this area as possible.

Fast forward a few years, and the launch of ChatGPT coincided with the start of my graduation project at college: both happening in November 2022. I always knew I wanted to do the project in some subarea of AI, but up until then I had thought about going for AI in games. After some discussion with professors, I ended up deciding to do it about applying BERT (and older LLM by Google) to the task of poem classification. Changing the focus to the intersection of AI and NLP was serendipitous, as I confess I had never heard about LLMs or the brewing revolution that was picking up steam at the time. However, with this change I had the opportunity to follow closely the arrival and consolidation of generative AI as the defining technology of our times.

A Thousand Nights And A Night

A big part of the fascination with generative AI is that it truly feels a little magical at times. Even to experienced professionals, specialists and the people who know intimately the mathematical and technical details that make it work. It is something so new and unexpected that everyone must have a period of adaptation and reframing of their expectations with regards to what these computer systems do (these are computer programs that can write human prose and suck at mathematics! the exact opposite of what computers have been doing since their invention).

In the very beginning, the book says that it takes 3 sleepless nights to get to know AI. I disagree. I think it takes at least a thousand nights to achieve that. And I fear I might even be setting an overly optimistic estimate.

I know I am getting close to those thousand (about two years of experimenting, which is roughly in the 700s days range) and I feel like I have barely scratched the surface of what is possible with this technology. The book itself describes several techniques I had not used, and that I have incorporated since then. And as newer and more powerful models appear, crossing into different modalities such as vision and audio, the possibilities increase faster than what anyone can cover.

Summary of main ideas

Before we get too dizzy talking about the exponential amount of possibilities that generative AI brings, let's take a look at the book's main ideas. These serve as the theoretical foundation for all of the experiments and wild extrapolations discussed as the book progresses.

General Purpose Technologies

Any technology always has some impact in a society, and usually in the society's economy. Some, however, are broadly-applicable enough to have a significant impact on the entirety of the economy. Classic examples are the steam engine, electricity and computers themselves - each of which completely changed the game when it entered common usage. The book claims that Artificial Intelligence is an instance of a General Purpose Technology, and it is very hard to argue against that. We have already seen AI, either the new generative deep learning trends or more classic approaches, being applied to a vast amount of fields of our society, with several more expected to also be impacted in the near future. As a side note, the fact that General Purpose Technologies is also abbreviated GPT seems to be totally unrelated to the flagship models that power OpenAI's ChatGPT - the earliest entry I found on Wikipedia's page about the topic, already using this very name, dates from 2014, four years before the first GPT models came along.

Four Principles

The book proposes 4 principles to keep in mind when interacting with AI systems. These are:

1. Always invite AI to the table.
2. Be the human in the loop.
3. Treat AI like a person (but tell it what kind of person it is).
4. Assume this is the worst AI you will ever use.

By following these, we have a strong framework to analyse our interactions with AI, and to have a sense of where we are going with them. And this is key: everything is happening so fast with this technology, that just understanding what is going on is a major concern for anyone. I will have more to say about this later, when I talk about my takeaways.

Centaurs and Cyborgs

Another separation that the book proposes is in two modes of working with AI:

• Centaur: when there is a clear separation between person and machine, with the human doing one set of the tasks and the AI doing another set. For instance: a human delegates to an AI system the summarization of the news of the day, which the human later reads to update himself on what is going on. The analogy that gives this mode its name is with the clear separation between the top side of the centaur (which is entirely human) and its bottom side (which is entirely animal).
• Cyborg: when there is a blending of the person and the machine, with tasks being handled together and the "pilot seat" alternating frequently between AI and human. A good example is how LLMs are being used for coding in several products that have come along the last years, with inline suggestions from the AI being fed in real time while the human codes. The analogy is with the fully integrated nature of cyborgs, organisms that are composed of both organic and electronic parts as equals.

Personal Highlights

Moving on to my personal opinions about the ideas in the book, here are some of the things that most drew my attention.

A Time For Explorers

There is an old bittersweet meme that describes us as those unfortunate beings that were "born too late for the sea, too early for deep space". I admit I have often felt like that myself: someone deprived of a vast and new field for exploration, where one could get lost in the promise of adventures and riches never before dreamed. Generative AI solves that.

At least for the moment, it is very clear that we are living the early days of something extremely disruptive. It is impossible to see humanity going back to how things were before co-existing with ubiquitous artificial intelligence systems, in the same way that once the internet was introduced it was impossible to go back to an offline world. While there definitely is a large layer of hype surrounding generative AI, and we have not even started to explore the truly disruptive implications of it (both things that were true of the internet in the late 90s and early 2000s), it is clear that this technology will be a foundational part of how society will function in the future.

The book effectively captures this feeling in several places. One of the direct implications of this is that the main task of each and everyone of us, right now, is to use these systems, explore them, explore our interactions with them, find out their strengths, weaknesses, breakthrough points, etc., as the ultimate capabilities and limitations of AI are currently unknown. And I agree with the argument the book makes about this being first and foremost a task for the individuals. Large entities, such as corporations and governments, will definitely experiment and find out how to employ this technology for their own ends, but by their very size this will be a long process. As individuals, we can experiment much faster, as we have no need to coordinate efforts along chains of reporting, to make strict allocation of resources to this effort, and we have very little to lose if we decide to just drop an experiment and switch to a new, more promising one, at any time. We get to be the explorers now.

Similarly, this mindset of experimentation and sharing brings the excitement associated with fabled moments of our past. We all learned to write good prompts by reading, copying and experimenting with the prompts openly shared by anonymous enthusiasts, who themselves learned much the same way. To the point in which sometimes it is considered good behavior to share your prompts together with your results, in a sort of "open prompt" philosophy that reminded me of the early days of computer programming, where people would learn to code by checking out pieces of code that other developers had created. I have tried to share my prompts whenever I have used a relevant amount of generative AI in my personal projects, such as for the creation of the User Stories for SnakeJS. The book talks about how generative AI differs from traditional software by not coming with an operating manual or tutorial, leading to everyone sharing prompts "as if they were magical incantations rather than regular software code", a beautiful turn of phrase that reminded me of Simon Willison's "We get to be wizards now!" moment.

All of a sudden we get to be both explorers and wizards!

Potential for education

Another point I enjoyed in the book was its emphasis on the impact that generative AI can have on education. I was particularly happy with how it explored both sides of the issue.

As a professor, the author is very aware of the negative impact that easily accessible AI has on the current mainstream mechanisms used by educational systems. As an intelligent professor, he is also very aware that the right way to deal with new ideas and technologies is not to fight back against its disruption of current models, but instead to focus on how its strengths can bring us to the next level by proposing new models that achieve more than the previous ones. The book describes a few of the experiments that the author used to include generative AI as a first-class citizen in his classes, and the outcomes.

Beyond formal and group education, generative AI can be a huge boost to personal learning and growth. The current state of the technology already brings us teasingly close (but not all the way!) to the vision of everyone having a personal tutor, mentor and guide to facilitate us following our interests. This model is more aligned to the classical view of education (think Ancient Greeks and their education system - at least for the elite), in opposition to the industrial-grade "factory of standardized citizens" model that represents the current state in most western societies now. Ubiquitous, personalized, generative AI systems can finally solve the problem of scale, which held us into the non-optimal model for so long.

As a final remark about education, the book also makes the counterintuitive argument that generative AI can mean an increase in value for an education in the Humanities. For a long time, more profitable careers and a higher status has been driving people away from the Humanities and into more hard-science fields such as Computer Science and Engineering. However, since generative AI models are trained with the accumulated knowledge our species has gathered in writing, images and sounds, a deep knowledge of the cultural heritage, artistic history and philosophical traditions suddenly means having a concrete advantage in extracting the best results out of these models. I fully agree with the book in that, I think if we are wise enough as a society we will start giving more value to the study of the Humanities.

AI success in exams

Although almost everything I have to say about the book is agreeing with it, there is at least one aspect in which that is not the case. The book, as almost every other source that talks about current generative AI models, talks highly of cases in which AI models (most often LLMs) achieve high results in exams and tests commonly used to assess humans. I have to say that these cases do not impress me too much.

The cynic in me has always disregarded almost every single test I saw as a very weak proxy to evaluating real knowledge. Be it an university entry exam, a cheap IQ test or any other evaluation mechanism, they always end up confusing some operational skill with knowledge: dissertations test your writing skills, not your knowledge (much less your intelligence); multiple-choice exams test your memory skills, not your knowledge (much less your intelligence). A test only tests the subject's ability in passing the test, never the subject's knowledge or intelligence. I have passed way too many exams I was hopelessly underqualified for during my school and college years to take any such success by AI systems seriously. They have much more interesting, useful and mindblowing proofs of intelligence than passing tests.

Comparing to expectations

Finally, for each book I study I always like to make a comparison of how it stood up to my expectations. In terms of quality, CoIntelligence completely satisfied my expectations. In terms of usefulness, it by and large exceeded them. I have found in this book both inspiration and specific techniques to incorporate more AI into my daily routine.

One significant difference between me and the book is that the book focuses on frontier models, that is, the biggest and most powerful models currently available, all of them hosted by some big corporation (OpenAI, Anthropic, and so on). I am more interested in local and open models, that anyone can run on their own devices. While I understand that there is indeed a big gap in cognitive performance between these two types of models, I personally think that the customizable and private nature of open models makes them more disruptive, as they are the ones that can truly be incorporated as a personal extension of one's cognitive capabilities with no threat to individuality. I was aware of this difference in interest before I picked up the book, though, and found that it had no negative influence in my reading.

Conclusion

To wrap up, I think CoIntelligence is an outstanding book, full of interesting, inspirational and useful ideas. It is specially useful for those who are not constantly reading every piece of news around this topic, but that still want to have a solid framework with which to reason about the usage of artificial intelligence systems.

In the foreword to the book Working Effectively With Legacy Code, Uncle Bob highlights how he enjoyed the way Michael Feathers talked about having that "... then it began" moment in programming. That moment when you write a piece of code and, for the first time, you realize you will never stop doing it for the rest of your life. I think CoIntelligence captures very well the similar feeling that so many people are having with AI, that feeling of having just met the Future and wanting to dive deep into it as much as possible. It provides excellent guidance as we move forward and, together, by experimentation and open collaboration, learn how to co-exist with our new found partners, and what our shared future will look like.

And it reminds us that our nights have only just began.

Project: JenAI

Description

JenAI is a command-line interface tool to chat with a local LLM, written in Java.

I wrote a sketch version of it around May of 2024, and released an official version in August 19th, 2024.

The application assumes you have an LLM server running on your machine (this is a deliberate choice), by default on port 8080 but that is configurable. Once started, the application displays a short introductory message in the terminal and starts a conversation loop between the user and the LLM, until the user chooses to exit. The personality the LLM is asked to assume is simply that of a generative AI chatbot whose name is inspired by its interface being implemented in Java, which should make it generic enough to be adapted to any purpose (I am strongly inclined to make the starting personality configurable as well, eventually).

As there are plenty alternatives that are much more advanced and reliable (such as Simon Willison's llm), I intended JenAI to be mostly of use to myself, as a learning tool.

Context

I developed JenAI as I was exploring how to build systems using generative AI technologies for the first time. I blogged about the set of 4 projects that came out of this exercise in my post entitled "First Steps Into AI Engineering". JenAI was the first of these 4, and as a consequence several of the things I learned and developed as I created it went on to play a part in the following ones.

Being a part of this context, for JenAI I wanted to have as much hands-on coding as possible, while still staying strictly away from training or fine tuning any model myself. I was, and still am for the foreseeable future, far more interested in the area of AI Engineering (using AI tools as a component to be incorporated in a software system) than in the core Machine Learning task of actually developing these models. As the canonical advanced way to consume the frontier models is currently through their APIs, adopting the same architecture and relying on an LLM server offering a reachable API seemed like a logical choice. The focus on models running locally is a manifestation of my current interest: I want to know what we can achieve with models that are open, private and personal - this technology is way too powerful to be monopolized by any centralized entity.

As a consequence of it being a project in which I wanted to do as much relevant hands-on coding as possible, in JenAI I avoided using frameworks such as Spring AI and LangChain4J, so that I had full control to experiment, try out things, and understand the details of interacting programmatically with generative AI models.

Highlights

While this approach has given me a lot of insights, it also came with some downsides. The client class responsible for making the request to the server and parsing the response, for instance, ended up with too much low-level wrangling that is way too prone to encountering problems and crashing. I even had to write a separate utility method to sanitize any type of special character that could crash during serialization - which means that conversations always reach the LLM with altered texts ("n" instead of "ñ", "c" instead of "ç", "ss" instead of "ß", so on) that can lead to degradation in the quality of the answers. As this is not really a core part of the system, I intend to replace this with a third-party library to handle the request/response communication in the future.

Another issue caused by the design decisions I made for the project is that the simplest way I could get an user input through the terminal in Java was to use a Scanner. This fits perfectly well with my initial intentions for the project, but it leads to horrible user experience. For instance, pasting text only works for single-line fragments - pasting anything with a line break makes the application trigger multiple request using only parts of the total fragment. Another problem is that using the directional arrows do not work for moving the cursor when writing a message, forcing the user to use the backspace key and delete anything that came after the part which they want to change. Now that the project is functional and its initial goal has been achieved, I plan to investigate better alternatives for this (as someone who rarely writes CLI programs in Java, I am not familiar with the best practices on how to handle complex user input from the terminal), probably by choosing a third-party library to handle user input as well.

On the bright side, several of the classes I developed in JenAI proved capable enough to be reused in the other projects I did. Among others, this includes both the LLM client with all the issues I mentioned above and the classes that model the conversation itself, with these having no issue whatsoever. So the project really worked as a stepping-stone for more complex ones.

I have found JenAI surprisingly useful since I finished its first version. Because I knew it from inside out and had total control on how to customize it, I was able to incorporate it on my local setup in a way such that I am always only a keyboard-shortcut-press away from launching it and having a helpful LLM assistant to discuss anything. And this led me to increase significantly the amount of time I spend interacting with LLMs, as the friction of having to launch a browser tab and navigating to a page to get started was usually enough to drive me away from doing it. I have been using JenAI to, among several other things, get very specific song recommendations ("You know that song Ma Baker, by Boney M? What other songs like that would you recommend? [...] No, not just any disco songs, I mean songs roughly from the 70s with a good groove, great narrative lyrics and a deep personal story with few cliches and a powerful plot"), cheer me up with a grumpy-developer joke on Monday mornings and get tips on how to better monitor Linux system's resource usage. One specific example are the first two issues I created on JenAI's own Github repository (Issue #1 and Issue #2), which I got nicely formatted in markdown and with good descriptive text by simply passing a loose ten-word summary of the problem and asking JenAI to format it using the best practices for Github Issues. Could I have done all of this using a production-grade alternative? Of course! But using a tool I created by myself to do it gave me an amazing sense of purpose and accomplishment.

Future Expansions

I already mentioned some of the things I intend to improve in the future, such as a better user experience for user input, a more robust mechanism for the API client and making more of the parameters customizable (personality, for instance).

Another improvement to user experience that I plan to do is implementing streaming text for the LLM answer - if the model being used runs slowly for any reason it is really frustrating to be a long time looking at a terminal window with no indication whatsoever of what is happening. I don't expect this to be an easy change, though, so it is something that might take me quite a while to figure out how to do well. Ideally, I want to have both streaming text and the ability to stop the answer midway by interrupting with some keyboard shortcut, to avoid being locked waiting for an answer that has already started wrong.

Yet another improvement I am planning is the ability to save and load conversations, so that a chat can continue over multiple sessions. I expect this to be fairly easy to achieve, as the conversation itself is modeled in a way that should make it easy to serialize and deserialize.

Setup

Although JenAI is not one of my portfolio projects (which have a fixed set of quality standards I expect to maintain through their entire lifecycle), I did configure most of the foundations I use for those.

I use Github Actions to generate a new release for JenAI whenever new code is pushed into the main branch and alters relevant core files of the project.

I have both a changelog file and an architecture.md file (an idea I adapted from this great article), beyond the usual readme file as documentation.

The only major thing missing in comparison with the standards from my portfolio projects are automated tests. Usually, I find these indispensable - Michael Feathers said in Working Effectively With Legacy Code that his definition of "legacy code" is code without automated tests, I consider automated tests the difference between amateur code and professional code. However, for the purposes of the projects I am doing while first exploring AI Engineering, I have chosen to not implement tests when I first develop them. I chose to do so because I feel that, in this exploratory mode, automated tests become a burden. The ability to test something comes from your knowledge about this thing, about what it should and should not do in a very precise and unambiguous manner, for the relevant scenarios. I simply do not know yet enough about the subject to effectively write tests for them, in fact, I am developing these projects exactly to learn more about it. So, while I am in this exploratory mode, I will restrain myself from writing tests and just explore.

Links

Source code: Github

Executable: Releases

First Steps Into AI Engineering

Generative AI is the biggest development we have achieved in our generation.

Nothing so transformative has appeared since mobile phones and the cloud - and, arguably, generative AI has the potential to have a much bigger impact than either of those in the years to come. It is very exciting to witness something like this happening first-hand.

My interest in the field

I have followed the entry of this area into mainstream discourse for a while. I started last year, 2023, working on my graduation project at college, in which I applied an older LLM (Google's BERT) for a specific task (poem classification), so I had just been studying and understanding how these models work internally when they exploded in popularity. For the rest of that year (and the first part of this one) I was too busy solving all sorts of issues, but I tried to keep up with the news, reading regularly whenever something came up and trying stuff out whenever time allowed it.

Now that things are more stable, I have started exploring the use of these technologies for building systems. While I am not particularly interested in training models and all of the Machine Learning / Data Science part of the endeavour, I am very interested in how we can take these models and use their capabilities in ways that expand what we can achieve with computing systems beyond what has been possible until now. This is what is being called AI Engineering.

So in the past three months I have been building a few projects to get familiar with incorporating generative AI models (mostly LLMs, but also image generation models) into deliverable systems. These were mostly exploratory work, private PoCs and experiments meant for practice. Now that I have learned a few things by building them, I am making public and open versions of them to leave as reference. I expect to work on this for a couple of months, as I create public repositories for each on Github and polish both the source code and the set of features available.

Axioms

There are a few things I have decided to follow in these projects. These things are very important to me, at this particular moment of time. Some of them might change in the future, but, for now, I feel comfortable in making them axioms for this initial set of projects. They are:

1. I want the projects to work specifically with local models.

2. I want to build these projects using Java.

3. I want the projects to be as minimalistic as possible, avoiding ready-made frameworks like Spring AI or LangChain.

Of these, I consider the first one to be the most important. At the moment, my interest is very much into what we can build using local, private, personal AI models. Even though most of the "frontier" models are closed, I believe the real transformative nature of this technology can better express itself when it is in the hands of the final users, in a similar way as what happened when personal computers became ubiquitous, breaking through the model of large mainframe servers controlled centrally by some entity. I might change the focus of my interest with regards to this in the future, but I don't expect that to be very likely.

The second and the third ones are due to more practical concerns: I want to build them with Java because it is my main professional language and using it will serve as a double exercise; I want to avoid frameworks because I want to build things by myself as much as possible in this first stage of learning. Both of them might very well change in the short or middle term future, but they are set in stone for these specific projects.

Scope

I came up with 4 projects to develop as the scope for these first steps. Each one of them allows me to explore a new aspect of the technology in a more or less progressive manner.

Three of those four I have already developed in a sketch-y manner, I intend to recreate open versions of them from scratch while making their code cleaner and more organized. I haven't included in the scope making them usable to a large audience, as their purpose is still to be exploratory experiments to give me practice in working with generative AI technologies, instead of becoming general-use tools for a community.

The fourth one I have not yet developed, but I have a clear enough idea of what I want with it to be confident that it should not take too long to code up.

I intend to write a blog post for each one of them as I publish their public versions. I will also update this post with links to each one during the process.

Projects (ongoing)

1. JenAI.

The first project is the simplest application one can think of for using LLMs: a chatbot. And in the simplest platform a developer could think of: the terminal. This project allowed me to learn the basics of consuming LLM models through API calls, and managing the state for a continuous conversation. I am having fun using it personally both for silly humorous purposes as well as serious work and learning, but its scope will never grow enough to match the production-grade alternatives, such as Simon Willison's llm.

Source code: JenAI @ GitHub

Blog post: JenAI @ Blogger

2. Chargen.

The second project is a desktop application to generate both avatar pictures and biographies for fictional characters. This project allowed me to work with two different types of generative AI models within the same application (it was also the first time I worked with Stable Diffusion programmatically). It required a different approach to prompting, one that used more specific prompts that would be used only once (so no state handling) and with a very specific type of expected output.

Source code: Chargen @ GitHub

Blog post: Chargen @ Blogger

3. Local Language Practice (LLP).

The third project is a desktop application to practice languages through roleplaying a conversation between two characters. This one was considerably more complex than the two before it, and made me think about and iterate through the main prompts (especially how they would be built using extra information from each scene) with much more care. It also allowed me to add an extra usage of LLMs, as a built-in translator widget to help maintain the flow of the conversation while clarifying any part that the user does not understand. It was quite interesting to integrate both usages in the same system, as it allowed me to think about LLMs in a more abstract manner, as components of an application instead of the entire application itself (or at least the overwhelming core).

Source code: LLP @ GitHub

Blog post: LLP @ Blogger

(This post will be updated with the other projects as they are released.)