Thinks & Links | March 9, 2024
BD&A - Thinks and Links
Big Data & Analytics - Thinks and Links | News and insights at the intersection of cybersecurity, data, and AI
📫 Subscribe to Thinks & Links direct to your inbox
Happy Weekend!
Being Together
It was great seeing old friends and new at the Sales Kickoff last week in Denver. At a large, remote-first company, there are people I’ve worked with for years that I’ve never met in person. While I largely appreciate and enjoy the benefits of working from home, there is nothing that compares to being in person with people you are working with. I believe there’s a number of reasons why this is true (and it ties back to where AI Security is going)
Body Language - meeting with someone in person gives a lot more information for where communication is happening and not happening. We’re able to tailor what we say and how we say it with the help of the feedback we get from being in the room with someone.
Relaxed Timelines - running from meeting to meeting to meeting puts a constraint on what we can talk about and what topics we cover. Getting a meal or sharing a non-scheduled conversation
Shared Context - coming out of an information session or hearing a speaker gives us something to respond to that activates different memories and thought patterns. This can lead to new insights and connections between ideas that we wouldn’t get over a zoom call
Interruption - It’s a lot easier to stop a conversation going in an unproductive direction when you can see and understand the misalignment in person. Steering discussions towards productive uses is much harder when you’re on a call
Physical Space - Aligning a new physical space to a conversation creates better memory patterns and stronger recall. “remember that in Denver” versus remember our Tuesday 3:30 pm call.
I’ve been thinking about these things, because they’re all lacking from the current iteration of Generative AI experiences. People I speak to are either using GenAI for every menial task they can think of or have largely shrugged off the value. A chatbot is not the most inspirational interface. But the foundational models that power chatbots make possible new experiences that can create more responsive digital experiences. Multimodal AI can process image and inputs that represent group dynamics. It can also bridge the gap for when meeting IRL just isn’t an option. Information retrieval techniques enable the creation of shared context and information about a topic that is timely, useful, and unrushed.
Imagine a not too distant future where I have an AI assistant that knows all about my work, my recent experiences, and my challenges. How might I engage with that assistant to help find the people in my company and network who can help me solve a problem? What if it could reach out to me proactively when there are things it knows I need. Perhaps my agent talks to your agent, and they collectively agree that we really should talk. Or all our agents share information about the meetings that happened today and help to plan and contextualize the follow ups. One day soon you might even have an AI that can whisper in your ear before each meeting to help you prepare:
Today, many organizations are beginning to explore the capabilities of Copilots that can access various kinds of data and automate some tasks. Their utility depends on the amount of data that is shared with them and the ingenuity of the prompts users give them. They’re also just the very beginning. Copilots are the first iterations of a long roadmap that includes greater autonomy and proactivity. They’re getting better and cheaper all the time. Many technology companies and startups are focusing on variations of this capability. OpenAI is reported to be focusing on this capability. Breakthroughs in this field now will find their way into business software over the next 12-24 months.
Agents that provide more and more utility will soon have access to read and influence a lot of data and processes. Most businesses are working through how to secure and productionalize their first proofs of concept involving LLMs, but they are far from ready for this level of AI Agent workflow. But the value is clear: paying for a global workforce to travel periodically gets expensive. Enabling AI to streamline human interactions and communications will make terrific economic sense.
It’ll be on us - the readers of this newsletter and those like us - to think carefully about the guardrails and monitoring necessary to capture the benefits and mitigate the risks.
Klarna Cuts Deep
Klarna recently announced that their AI assistant has handled over 2 million customer service chats, nearly two-thirds all such interactions, within its first month. The chatbot is performing on par with human agents. It has also significantly reduced customer wait times from 11 minutes to under 2, and improved issue resolution, resulting in a 25% drop in repeat inquiries. Available in 23 markets and over 35 languages, it’s expected to boost Klarna’s profits by $40 million in 2024, enhancing customer service and financial management experiences.
Klarna’s results along with their significant cuts of over 700 roles in the customer services department is sure to catch attention. Customer service has long been in the crosshairs of AI and automation, and the advances in Large Language Models make the provisioning of these capabilities significantly easier and less expensive. When the potential benefit is measured in multi-millions, security concerns should not halt progress, but rather a small portion of that profit should be used to mitigate new risks.
Claude 3
https://www.anthropic.com/news/claude-3-family
(Written by Claud’s Opus model)
Anthropic’s Claude 3 AI model family is setting new benchmarks in the AI landscape, with its most advanced model, Opus, outshining GPT-4 in tasks ranging from coding to common knowledge. Opus, along with its siblings Sonnet and Haiku, offer businesses and developers a range of options tailored to their needs. Opus excels at complex tasks, while Sonnet balances performance and cost, making it ideal for enterprise workloads. Both models can adhere to brand voice and guidelines, ensuring consistent customer experiences. The family’s enhanced vision capabilities allow for processing photos, charts, and diagrams, opening up new possibilities for AI-powered applications.
For the security-minded, Claude 3’s introduction brings both opportunities and challenges. The expanded API offerings, including the Claude Pro service and integration with Amazon Bedrock and Google Cloud’s Vertex AI Model Garden, provide new avenues for developers to integrate AI capabilities into their applications. However, the varying cost structures and performance levels across models may require careful consideration when balancing budget and security requirements. As the AI ecosystem grows increasingly complex, with multiple models and APIs in play, ensuring the security and integrity of AI-powered systems will be more critical than ever. Developers and security professionals will need to stay vigilant, monitoring for potential vulnerabilities and ensuring that the appropriate safeguards are in place to protect sensitive data and maintain trust in AI-driven applications.
Hidden Layer’s AI Threat Landscape Report
https://hiddenlayer.com/threatreport2024/
The AI Threat Landscape 2024 report unveils a daunting new frontier for security teams. With 77% of surveyed companies already stung by AI breaches and 98% betting big on AI for success, the stakes have never been higher. Adversarial attacks, generative AI gone rogue, and third-party integrations turned trojan horses – it’s imperative to incorporate AI into existing security programs.
To stay ahead of the curve, organizations need to get proactive. That means conducting AI-specific risk assessments, implementing best practices, keeping a watchful eye on models for anomalies, and having a robust incident response plan at the ready. The landscape may be shifting, but with the right strategies and a healthy dose of vigilance, security teams can navigate this new terrain and unlock the full potential of AI – without sacrificing the safety of their systems and data.
100 Malicious AI Models
Data scientists are the target of malicious machine learning (ML) models on Hugging Face, a popular model and dataset sharing community. JFrog’s research details the kinds of compromised AI models that can be found on the site and the attacks they enable - including unauthorized code execution leading to potential data breaches and system control. Many of these threats leverage the ‘pickle’ file format, allowing attackers to execute arbitrary code upon model loading, a method often overlooked in the AI community. Despite Hugging Face’s security measures like malware and secrets scanning, vulnerabilities remain, exemplified by the detailed analysis of an offensive PyTorch model uploaded by a user ‘baller423’.
We Hacked Google AI for $50,000
https://www.landh.tech/blog/20240304-google-hack-50000/
A fun story about three top hackers participating in Google’s “LLM bugSWAT” event to uncover AI vulnerabilities and earn bounties. From an Insecure Direct Object Reference (IDOR) flaw in Bard’s Vision feature that exposed private user images, to a denial-of-service vulnerability via directive overloading in Google Cloud’s GraphQL API, and finally a clever prompt injection attack that exfiltrated sensitive Google Workspace data, these researchers left no stone unturned. Google provided them access with top engineers from the AI projects so that hackers could ask questions and uncover vulnerabilities faster. The team even earned a 50,000 in bounties and a wealth of insights into the evolving world of AI security.
Demonstrable AI Worm
https://sites.google.com/view/compromptmized
Researchers have created the first computer worm that targets the growing ecosystem of GenAI applications. Named Morris II, this zero-click malware uses adversarial prompts (inputs crafted to intentionally fool or manipulate a model) to infect AI models, causing them to replicate the malicious input, perform harmful actions, and spread the worm to other agents. The team demonstrated the worm’s effectiveness against GenAI-powered email assistants, showcasing its ability to spam and exfiltrate personal data. As companies eagerly integrate GenAI into their applications, this study highlights the urgent need for robust security measures in the face of emerging AI-driven threats.
Safe Harbor for Researchers
This paper proposes that AI companies should provide legal and technical safe harbors to protect independent researchers conducting good faith AI safety evaluations and red teaming. The authors argue that current terms of service and enforcement strategies disincentivize important safety research due to fears of account suspensions or legal repercussions.
While some companies offer limited researcher access programs, these are seen as inadequate substitutes for independent research. The paper calls for AI companies to commit to not taking legal action against researchers complying with responsible disclosure policies, and to avoid penalizing accounts engaged in good faith research. These safe harbors are presented as necessary steps to enable more inclusive community efforts to identify and mitigate the risks posed by generative AI systems.
OpenAI Drama - Matters Less
https://www.reuters.com/legal/elon-musk-sues-openai-ceo-sam-altman-breach-contract-2024-03-01/
It would be strange to write a newsletter about AI and not at least mention the drama between Elon Musk and OpenAI. However, unlike a year ago, in 2024 there are many good alternatives to direct reliance on OpenAI. While ChatGPT and GPT-4 are among the best AI tools available, the outcomes of various lawsuits will not impact the AI ecosystem. It’s important to remember there are many alternatives and a growing number of very capable alternative models to use. ChatGPT ushered a new age of interest and investment in Generative AI. Time will tell if OpenAI itself will survive all of the heat it is drawing, but AI is here to stay.
Have a Great Weekend!
You can now chat with the newsletter archive at **https://chat.openai.com/g/g-IjiJNup7g-thinks-and-links-digest