Tech bro

Integrating work with AI assisted tool

For over a year or so since the launch of ChatGPT, we have seen many people’s work and lives disrupted for better or worse. As an interviewer myself, I have encountered many cases where candidates bluntly copy and paste GPT answers. Too obvious, the tone and structure are easily distinguishable.

I myself have also been exploring AI tools (the frugal way of course).  Using company-backed Google Bard, and now Gemini for minor tasks and opinions on my work.

We see many start-ups and tools pop up along the way. Many of them are behind paid-wall, it is fun to try with their free trial. But, what’s the point if I am sure I am not going to commit the tools for the long run?

My paradigm shifted about LLM

That’s what got me learning more about LangChain and prompt engineering. So that, I can achieve what I need with the existing available tool. If you’re into or want to get started in LLM, you definitely should start with these 2 courses.

Since then, I feel that my conversations with LLM are much more structured. I am able to control the direction and outcome. I even apply it with BBIT (I talk about it in my other post) using the Tree-of-thought process, as my sparring partner to generate tons of assumptions to work with. It helps to open up the doors and parameters for my team and PMs to experiment with the options.

Testing with Local LLM

Local LLM is also getting popular. With the release of Llama 3 which have on par performance with GPT-4 or Gemini Plus. No doubt, is worth giving a shot at this, right? Also, many LLM enthusiasts don’t want to get into vendor lock with the big names, without a doubt there will be a lot more progress can be seen from this direction. That’s where you see fine-tuned products on HuggingFace.

Is easy to get started with local LLM this day too. The light of Ollama and LM Studio helps onboarding and using it so much easier. But with a catch! You’ll need decent enough hardware to have a decent feedback time lag to deal with. Sadly, my machine is no match for this.

LLM services on the cloud

Truly blessed to be living in this era. There is no short of options. Keeping it short, there are others to choose from with decent free options. Eg. Anthropic or OpenRouter. After some exploration, I think everyone should give Groq a go. We’re getting it free maybe because is still in beta?

You get a pretty decent intelligence from a 70B nodes Llama3 to chat with. And the response is so fast that it makes chatting with LLM so much more enjoyable.

LLM interface with AnythingLLM

Finally, the last piece of the puzzle. You’ll need an interface that can make you come back for more and ask questions. There are many open-source tools – see the GitHub trending, and you’ll know. I’ll just spill the bean here instead. Eg. Lobe and my go-to AnythingLLM.

AnythingLLM – workspace

Is so much better compared to ChatGPT or Gemini. My main feature in this is the workspace. I can start to preset system prompts for easy access in the future. And, there are tons of examples available. Again, see GitHub trending ;). Fellow developers, these GPTs is strongly recommended.

The thought behind the prompt makes so much sense, and it has been helping me greatly with my PHP development for my stock research tool. I have extended out to many variations that I am using for my own. Eg. AWS Senior Engineer, PHP Senior Engineer, and so on. By doing so, I can skip all the framing for LLM and go straight into the question and get my expected output.

AnythingLLM – vector store

The next cool feature is the embedded vector store. In simple terms, this is the database to store data for LLM to use. I don’t need to dive too deep into this, go see their website to learn more on your own. This clicked for me because of the nature where I can upload a bunch of huge contexts (and can be reusable) for me to chat with LLM.

For example, I wrote a script to extract out price target of the stock analysis done by individual firms. And this allows me to ask a whole bunch of things instead of clicking and reading each article.

Conclusion

I think we are in an era of change. Learn to get comfortable with the trending tech. Is time to up our productivity or get left behind?

There is no excuse honestly. Imagine, all the things I mentioned above, you can get it all for FREE. Take the opportunity to learn for free while is still available. The quicker you onboard, the easier to keep up with the progress and updates.