Digits at Google I/O'24: A Fusion of Innovation and Collaboration

The Google I/O and the Google Developer Conference held in Mountain View, California, have always been a beacon of new technology and innovation, and 2024 was no exception. Like last year, Digits had the privilege of being invited to participate in this global gathering of ML/AI experts. Our team of engineers was thrilled and honored to be a part of such a dynamic and forward-thinking event.

Engaging with the Developers Advisory Board

One of the key highlights for us was participating in Google’s Developer Advisory Board meeting. This not only provided us with a platform to share our insights but also allowed us to exchange ideas with Google's Developer X group and learn about upcoming products.

A Closer Look at Google’s Innovations

From Digits' perspective, several announcements and tools stood out, each promising to significantly impact our journey with machine learning and artificial intelligence. Here’s a rundown of the highlights:

Gemma 2: A Leap Forward for Open Source LLMs

Google unveiled Gemma 2, a new model designed to enhance the capabilities of open-source large language models (LLMs). What makes Gemma 2 truly remarkable is its optimization for specific instance types, which will help reduce costs and improve hardware utilization. This is a significant advancement, as it enables more efficient and cost-effective deployment of ML models, a crucial factor for any tech-driven company.

Responsible Generative AI Toolkit

Another noteworthy introduction was Google's Responsible Generative AI Toolkit. This comprehensive toolkit provides resources to apply best practices for responsible use of open models like the Gemma series. It includes:

• Guidance on Setting Safety Policies: Frameworks and guidelines for establishing robust safety policies when deploying AI models.
• Safety Tuning and Classifiers: Tools for fine-tuning safety mechanisms to ensure that AI behaves as intended.
• Model Evaluation: Metrics and methodologies for thorough evaluation of model safety.
• Learning Interpretability Tool (LIT): This tool enables developers to investigate the behavior of models like Gemma and address potential issues. It offers a deeper understanding of how models make decisions, which is crucial for transparency and trustworthiness.
• Methodology for Building Robust Safety Classifiers: Techniques to develop effective safety classifiers even with minimal examples, ensuring that AI systems can operate reliably in diverse scenarios.

LLM Comparator: A Visualization Tool for Model Comparison

The LLM Comparator is another brilliant tool that grabbed our attention. It is an interactive visualization instrument designed to analyze LLM evaluation results side-by-side. This tool facilitates qualitative analysis of how responses from two models differ, both at example- and slice-levels. For engineers and developers, this means more insightful comparisons and a stronger ability to refine and improve their models.

Reflecting on Our Experience

Being invited to Google I/O once again, especially being part of the Developer Advisory Board meeting for the second consecutive year, is a testament to the growing partnership and mutual respect between Digits and Google. We are thankful for this opportunity and excited about the collaborations and advancements that will emerge from these engagements.

Our time at Google I/O’24 was not only inspiring but also a powerful reminder of the incredible pace at which technology evolves. With tools like Gemma 2, the Responsible Generative AI Toolkit, and the LLM Comparator, we are on the brink of a new era in AI and ML development. At Digits, we look forward to integrating these innovations into our work and harnessing their potential to create transformative solutions.

Big thanks goes out to the Jeanine Banks and the entire Google team for hosting us at the Google Developer Advisory Board meeting.

^* Image credits: Google

In April, Digits' expert machine-learning team was invited to conduct a lecture at the University of Washington. The event occurred at the Foster School of Business and was attended by a mixed crowd of students and faculty alike.

75 students flocked to the lecture, demonstrating the growing interest in these ground-breaking technologies, such as machine learning, that are paving future paths in finance. Undeniably, the turnout indicated the growing curiosity about practical applications of machine learning in the world of finance.

The lecture provided an overview of machine learning and Generative AI (GenAI) and explored their impacts in the finance sector. Attendees delved deep into understanding GenAI's specific use cases in finance, with our team sharing their exhaustive research findings and experienced insights to provide a wider perspective of GenAI's potential role in revolutionizing traditional accounting methods.

The University of Washington's proactive approach in inviting the Digits team and the hearty attendance underlines the increasing investment and gravitation towards AI technologies in finance. This trend is expected to continue as technology continues to weave its way into the world of finance.

In case you missed it, you can access the lecture slides below to help better understand this technological revolution.

We're excited to share the highlights from our recent participation at Google Next’24 on April 9 and 10, where we showcased Digits at the NVIDIA booth. This event provided us with an unparalleled platform to demonstrate our cutting-edge machine learning models, which included the first in the world to handle double-entry accounting effectively. This is a product of our robust partnership with NVIDIA, which we are happy to highlight today.

Our collaboration with NVIDIA, a leading powerhouse in GPU technology, has been instrumental in powering Digits' machine learning initiatives. With NVIDIA's support and vast tech resources, we have been able to build a state-of-the-art, secure, and private machine-learning infrastructure that has revolutionized the way we handle our double-entry accounting system. This partnership signifies an important milestone in our journey of harnessing machine learning to solve real-world business problems.

Our sessions at the NVIDIA booth offered us a unique opportunity to meet and engage with our current customers. It was a privilege to demonstrate how Digits supports startup founders by simplifying their financial processes and helping them understand their financial health. Feedback from customers during these sessions reaffirmed the benefits of our solutions in assisting startups in managing their finances with greater ease.

In addition to showcasing our technology, Google Next’24 was a fantastic opportunity for us to connect with Google experts. These interactions enabled us to gain valuable insights and learnings that we hope to incorporate into our future projects.

We are also excited to dive deep into state-of-the-art open-source machine learning projects at Google Next, like Gemma and JAX. These tools hold significant potential. Stay tuned as we will share more details on this in our upcoming blog posts.

In conclusion, our participation at Google Next’24 reinforced some of our fundamental beliefs - that collaboration fuels innovation, direct customer engagement is invaluable, and continued learning and exploration is a powerful tool for growth. We remain committed to leveraging the potential of machine learning to simplify business finances and believe that with partners like NVIDIA and platforms like Google Cloud, we are well on our path.

A special shout out is due to Michael Thompson, Bailey Blake, Matthew Varacalli, and Martha Aparicio from NVIDIA for this tremendous opportunity. We are already looking forward to next year's event.

Every year, Google invites customers and major product partners to their Cloud conference, Google Next. After a multi-year in-person hiatus, Google Next returned in full force to San Francisco’s Moscone Center, and Digits was invited to present how we’ve collaborated with teams at Google to create Digits AI.

Given our experience with Vertex AI across many ML projects at Digits, presenting at Next provided a unique opportunity to showcase how we have been working to push finance and accounting software forward, and also share our experiences in developing machine learning and AI using Google Cloud products.

🤖 Getting Early Access

In the weeks leading up to the conference, our engineering team received early and exclusive access to Google Cloud’s latest release of their Vertex Python SDK. This allows remote execution of machine learning model training or model analysis, all controlled via a local Jupyter notebook. In the coming weeks, we’ll share a more in-depth post, with detailed explanations and feedback on our experience using the new product. But for now, we’ve included a summary of our initial findings as well as a video of our talk at Google Next where we discussed our experiences.

Initial Learnings

Vertex AI has been a fundamental element in building lean machine learning projects here at Digits. We’ve outlined some of the various use cases which were also discussed in more detail during our Next talk:

• Vertex Pipelines → Any machine learning model in production is trained, evaluated and registered via CI-driven ML pipelines.
• Vertex Metadata Store → During the model training, any produced pipeline artifact (e.g. the training set, or the preprocessed training data is archived through the metadata store).
• Vertex Model Registry → Any positively evaluated, trained machine learning model produced by our machine learning pipelines is registered in a one-stop shop for future consumption.
• Vertex Online Prediction Endpoints → Data pipelines or backend APIs can access the machine learning models through batch processes or online prediction endpoints.
• Vertex Matching Streaming Enginex → Generated embeddings are made available through the embedding database service in Vertex, called matching engine.

Presenting at Google Next is an experience that outlines the true value of sharing information and learning from others in the industry. This event gave us a platform to share our knowledge with other customers and offer insights into our work and, conversely, we were privileged enough to glean wisdom from some of the industry’s most respected leaders in AI/ML as they shared their experiences and successes using Google products.

A special shout out is due to Sara Robinson, Chris Cho, Melanie Ratchford, and Esther Kim for this tremendous opportunity. We are already looking forward to next year's event in Las Vegas.

Digits engineers recently spoke at Google's North America Connect conference on the future of machine learning. This blog post expands on the presentation themes.

Over the past few months, we have witnessed groundbreaking developments in the field of generative machine learning (ML) models, revolutionizing the potential impact ML can have across diverse industries. Today, machine learning projects can be integrated with various applications in just a matter of hours, as opposed to the days or even weeks it took in the past. This not only saves valuable time, but also empowers companies to embrace technological advancements and drive innovation to market quickly.

As we attempt to understand the power of this rapidly evolving domain, we feel compelled to share our thoughts on the future of machine learning. Through this blog post, we aim to:

1. Dissect the intricacies of the field
2. Delve into the multifaceted aspects of generative machine learning via model APIs like OpenAI
3. Discuss the benefits and downsides that have the potential to transform lives of people around the world.

Has Machine Learning Found Its Gutenberg Moment?

When we think of history's greatest technological leaps, the invention of the printing press in 1450 by Johannes Gutenberg in Mainz, Germany, is undoubtedly one of the most transformative. Gutenberg's press revolutionized how books were copied and distributed, no longer requiring them to be painstakingly hand-written by monks.

This innovation significantly altered access to knowledge, becoming one of the cornerstones in history and leading to increased literacy and widespread access to information. The “Gutenberg Moment.”

Are we experiencing a similar revolution in machine learning, specifically within the realm of generative AI?

Similar to how the Gutenberg Moment democratized access to information, the recent acceleration in access to generative AI has empowered businesses to swiftly adopt previously inaccessible technology such as Large Language Models (LLMs) and foster innovation, moving the autonomy to work with ML outside the confines of large technology companies and closer to domain experts in various industries.

As generative models continue to evolve, it begs the question: Will this evolution redefine the core tasks machine learning engineers are performing? Instead of focusing on generating datasets, training and evaluating machine learning models, will we shift focus to engineering prompts for LLMs?

Early Lessons Learned

When we first interacted with large language models, we were in awe of the generated human-like text. However, drawing conclusions based on brief interactions with these models can be misleading. It's essential to be cautious of initial outcomes, as LLMs are capable of producing highly convincing "hallucinations" or fabricated information within their output.

Moreover, LLMs may generate inconsistent outputs, reinforcing the need for human review in employing them effectively. As we continue to explore the potential of generative AI, understanding and mitigating these limitations will foster progress and unlock more reliable and robust applications.

Is Machine Learning Commoditized?

For certain projects, machine learning indeed appears to be commoditized by the capabilities of large language models. Typically, these projects involve using ML models based on public data or ones that do not require specific environment settings (e.g., on-device processing). Additionally, projects without stringent security or privacy requirements can also benefit from accessible model APIs like GPT-4 or PaLM 2.

However, not all projects fit into this commoditized landscape. Projects involving proprietary data or ones with strict privacy requirements still tend to need custom-built ML solutions. This is because 3rd-party model APIs may not factor in the unique traits of proprietary datasets, require impractically long prompts, or don’t provide the necessary security measures. Furthermore, projects with low latency requirements may also necessitate specialized ML solutions tailored to specific use cases, as the development for low-latency inferences of LLMs continues.

The importance of the underlying intellectual property (IP) should not be overlooked. If underlying data and custom models can provide you an unfair advantage, it is worth protecting it and further investing in it.

Should We Be Concerned About Model APIs?

Over the years, the machine learning community has consistently focused on achieving unbiased predictions, improving data and training transparency (e.g. through model cards), closing feedback loops for better model performance, ensuring user privacy, and enabling on-device inferences. However, as we move toward adopting third-party generative AI and incorporating model APIs, it's crucial to be aware of the potential issues and challenges they may pose.

Currently, the desired objectives mentioned earlier are not completely achievable with model APIs. There are concerns that, unlike more traditional AI models, generative models like GPT-4 may be more susceptible to producing biased results due to their complexity and the vast amount of data they need to process. Additionally, essential privacy features may be compromised when processing user data via model APIs, since these frameworks often require transmitting data to remote servers.

Transparency regarding data and training is an ongoing challenge for model API developers. Industry-leading models may not fully disclose their inner workings, making it difficult for users, and even industry experts, to fully judge their ethical implications. Lastly, on-device inferences, which have boosted privacy and efficiency in the past, are currently impeded by the large size and resource requirements of sophisticated generative models. In summary, as we continue to integrate model APIs in the realm of generative AI, it is essential for the ML and developer communities to be cognizant of the potential limitations and risks associated with their use. To fully harness the advantages of such powerful technologies while adhering to the standard objectives concerning privacy, transparency, and unbiased predictions, researchers and practitioners must be diligent in addressing and overcoming these challenges to strengthen their contributions to the field.

How is the role of Machine Learning Engineers changing?

The responsibilities of machine learning engineers have expanded beyond solely developing models to encompass a wider range of tasks associated with generative AI systems.

One of the key changes in our role is to act as effective moderators between various stakeholders. This involves liaising with clients, leaders, and other team members to ensure that a generative AI project is well-executed and the stakeholders (e.g. software engineers consuming third party model APIs) understand the implications of the hyperparameters.

In addition to being moderators, ML engineers now serve as advisors regarding the risks and benefits of generative AI projects. We use our knowledge of the field to inform stakeholders about the potential outcomes and consequences of implementing a particular model, as well as to identify potential biases and ethical issues that should be managed proactively.

With these changes, ML engineers are transitioning from creators to consultants. The role is no longer focused solely on designing and implementing algorithms, but rather on guiding and supporting organizations in navigating the complex landscape of generative AI. This shift requires us to develop not only technical expertise, but also strong communication, collaboration, and critical thinking skills to address the challenges and opportunities that generative AI presents in various industries.

Conclusion

In conclusion, although prompt design plays a significant role in the development of generative AI, it does not eliminate the need for machine learning expertise in its entirety. As we continue to grapple with machine learning engineering challenges associated with large language models, it becomes increasingly important to have a deep understanding of ML for integrating concepts such as bias and safety effectively. To optimize the value of generative AI, organizations should focus on projects with proprietary data, those involving "subjective" machine learning (e.g., similarity machine learning), and those with specific requirements in user privacy, security, and low latency. As experts and advisors, finding the right balance and alignment among stakeholders is crucial to optimally navigating the opportunities and challenges posed by this emerging technology.