Although my strategies weren't always successful, my supervisors always gave me the space to learn from my failures.
The young STEM talent was not let down as his first assignment was to assist in the development of an AI tool that could potentially empower SPF officers. Called Wordsmith, this tool uses Large Language Models (LLMs) to generate and summarise meeting transcripts. Wordsmith also flags out key action items from the transcripts.
The use of Wordsmith would mean officers spend less time on administrative work and more time on pressing tasks.
Deciphering the complex system behind PromptFlow. (Photo: HTX)
While working on Wordsmith, Ignatius also got to learn about another AI tool—a state-of-the-art development platform for LLMs by Microsoft called PromptFlow. With PromptFlow, the xData team can rapidly experiment on LLMs and develop prompts that improve Wordsmith’s speaker distinction capabilities.
However, PromptFlow’s frequent updates by Microsoft meant that Ignatius struggled to find comprehensive guides that could deepen his understanding of the software.
Fortunately, HTX arranged for Ignatius to attend Microsoft's training sessions on deploying PromptFlow.
“I loved the programme as it gave me a better understanding of PromptFlow. This let me perform further tests and experiments to explore how various prompts worked within the Wordsmith model—it was like discovering a new method of exploring LLMs!” he shared.
Evolving expertise
Once Ignatius learnt how to work with LLMs in Wordsmith, he then embarked on a more challenging project involving another AI model called Vision LLMs.
The difference between these two AIs? While Wordsmith primarily uses transformer-based models for language processing, Vision LLMs contain both transformer-based models and convolutional neural networks, allowing the model to interpret and generate both visual and textual data.
“Working on these Vision LLMs reminded me of why I wanted to join HTX in the first place – this technology could greatly ease the workflow of Home Team officers as they can potentially help analyse multi-page documents in various reports to improve the efficiency and accuracy of administrative and help in verification matters,” he explained.
However, developers of open-source vision models often train their models on data that is not relevant to the Home Team. This meant that Ignatius had the opportunity to shortlist a few open-sourced Vision LLMs and finetune them to suit the Home Team’s needs.
“This was a challenging task because I had to understand how and why the open-source vision models responded differently to certain prompts and our domestic datasets. While it wasn’t easy, I enjoyed experimenting with different prompts and codes to get the exact result I wanted,” he said.
Ignatius hard at work fine-tuning a Vision LLM. (Photo: HTX)
Looking back at his internship journey, Ignatius said he is grateful that he has been given so much room to experiment with various AI solutions.
“My supervisors at xData have never restricted me to a single course of action and this really allowed me to explore various ways of handling difficulties in my project,” he said.
“Although my strategies weren’t always successful, my supervisors always gave me the space to learn from my failures. And that’s one of the biggest takeaways from this internship – failure really is the best teacher!”