Thinks and Links | July 12, 2024
BD&A - Thinks and Links
Big Data & Analytics - Thinks and Links | News and insights at the intersection of cybersecurity, data, and AI
đź“« Subscribe to Thinks & Links direct to your inbox
Happy Friday!
You Can’t Defend What You Don’t Understand
There’s been a lot of discussion by regulators, investors, and reporters in the technology space over just what is happening with AI. On one hand, there have been wave after wave of new innovations, companies, and capabilities. Presentations from OpenAI, Google, Microsoft, Apple, and all showcased incredible new benefits and AI superpowers for businesses and individuals. These companies’ revenue and stock prices have also increased substantially. On the other hand, the real adoption of AI and the reflection of these benefits on the bottom line remains mixed. Many companies are currently trialing MS CoPilot or using ChatGPT’s enterprise features with the hope that efficiencies will appear. In the public markets, it’s unclear that those benefits are showing up. Teams in companies of all sizes are building applications with AI APIs and/or local models. Many of these have added cost and risk but have yet to demonstrate value. There is growing uncertainty about what the true ROI will be for all this AI investment. There are signs that the AI Hype cycle is cooling.
Good. That’s when the real useful work begins. That’s where we have a critical role to play.
Business leaders regularly report compliance, risk, and security as blocking factors for not innovating faster. This was before the introduction of capabilities that could make judgements and take actions autonomously. The most critical business functions have the highest risk and likewise the highest scrutiny. But that’s also where the greatest value is. So, to reach production innovation needs security. “Data Science Fair” projects can be done all day, but significant ROI will only be possible on the other side of Securing AI. And the definition of Secure AI depends on who you ask.
“If you know the enemy and know yourself, you need not fear the result of a hundred battles” - Sun Tzu, The Art of War
“If you know the enemy and know yourself, you need not fear the result of a hundred battles” - Sun Tzu, The Art of War
Secure AI starts with knowledge. How well do we understand what attackers might do with AI enabled tools and the changed attack service presented by new implementations of this technology. How well do we know our own capabilities across the organization for deploying and protecting various forms of AI. When we know what AI can do and what risks it can bring, we have taken the first steps on an important journey.
That’s why I’m very excited that Optiv has launched industry-leading trains to help Security and Business professionals understand AI. For free. Available now.
Secure AI for Business Stakeholders
Cybersecurity leaders should take the “Secure AI for Business Stakeholders” course to learn how to effectively implement and manage AI technologies while addressing associated security risks and governance challenges in their organizations.
Secure AI Literacy and Awareness
Everyone should take the “AI Literacy and Awareness” course to learn more about AI technologies, their potential, and their risks to make informed decisions and drive competitive advantage in their organizations.
These free courses are a great way to catch up on the latest trends and details of Secure AI. Please share your feedback so we can make practical knowledge of AI more wide-spread and strengthen our defenses.
Claude 3.5 Makes AI Fun
https://www.anthropic.com/news/claude-3-5-sonnet
Lately, I’ve been having a lot of fun with Anthropic’s Claude 3.5 Sonnet. This latest release from the AI research firm outperforms GPT-4 on many tasks. I’ve been finding Claude to be more likely to produce a first draft of writing better than ChatGPT, and its coding tips are also working for my limited needs. But several new features have made the platform more useful for me.
The first is Artifacts, which allow you to request and review content to be generated in a window side-by-side with the AI. The interaction feels more dynamic than ChatGPT because artifacts also display the output of code that they write. In a split-screen view you can iterate and converse with the AI model to write code, produce a document, write webpage, build an SVG image, create diagrams, and build React software.
This is a lot of fun. I’ve been enjoying seeing lots of creative projects people have done with it. Someone built a small rocket landing game in the app:
It’s also a great tool for learning – by using the LLM to process large PDFs or explain complex topics, then using the application code writing to create interactive, educational dashboards:
And this week they introduced a way to publish your artifacts so that others can try them out. So here’s one that you can try at home: it’s a CISO simulator for Secure AI built by yours truly: https://claude.site/artifacts/cdf670c5-07e5-45b9-8d45-524cb8aa2541
Claude has also added a capability called “Projects” that lets you store files for use with the model across multiple chat sessions. This lets you centralize the information that is to be shared with the AI so that it can be quickly referred to rather than re-uploading each time you start a session. I use this to keep a writing sample ready to help me summarize large articles and blog posts every now and then…
These enhancements make Claude the AI of the moment. Working with it is fascinating and fun. Expect concepts like Artifacts and Projects to come to other AI Services and product features over the next few months.
Teamwork Makes the Vulnerability Attacking Nightmare Work
https://arxiv.org/pdf/2406.01637
Here’s a research paper about how LLM agents were used to make advances in Vulnerability exploitation. By using a group of LLM agents working together, researchers were able to show how a small benchmark of zero-day vulnerabilities could be breached. Autonomous vulnerability-seeking agents don’t appear to be here yet (at least if it is, no one is sharing) but a researcher working in tandem with these agents was able to speed up their vulnerability exploitation by 5x. The research is interesting, but the implication is that we can expect more and faster breaches in the future.
Challenges of Red Teaming AI Systems
https://www.anthropic.com/news/challenges-in-red-teaming-ai-systems
This article from Anthropic has a great overview of the various flavors of Red Teaming that they have been using to make models more safe and secure. There are many categories of testing that require different skills and resources:
Domain-specific (e.g., trust, safety, responsible AI, cybersecurity)
National Security (e.g., producing bioweapons or advancing nation state agendas)
Regional variations (e.g., multilingual, and multicultural variations)
Automated and Continual Red Teaming
Open-ended, crowdsourced, and community Red Teaming
While not all types of testing will apply beyond the AI research labs, it is a useful thought experiment to consider how these categories of testing align against your own AI testing and Red Teaming plans.
Federal Government’s AI Tabletop
In June, the Cybersecurity, and Infrastructure Security Agency (CISA) ran their first AI security tabletop exercise with 50+ experts from government and industry. They simulated an AI-enabled cyber incident to build an AI Security Incident Collaboration Playbook. Big names like Microsoft, Palo Alto Networks, and OpenAI joined in. CISA Director Easterly stressed “secure-by-design” is essential for AI system development, and the coordinated exercise is further evidence of the seriousness of this challenge.
Microsoft CoPilot for Security - Worth the $35,000?
https://campbell.scot/thoughts-on-copilot-for-securitys-early-days/
Copilot for Security is one of several generative AI solutions available from Microsoft - easily confused with the more ubiquitous O365 CoPilot. The Security version of CoPilot integrates with MS Defender XDR, Entra, Purview, and Intune. It also is much more expensive! While Copilot shows promise in areas like incident summarization, it struggles with some security-related queries and script analysis. The current pricing model, starting at around $35,000 annually for one Security Compute Unit (SCU), raises questions about cost-effectiveness compared to alternative security investments. Despite current limitations, there are promising developments coming on Copilot’s roadmap, especially if it can be provisioned on-demand and reason over wider data in the Microsoft Graph.
Finding and Mitigating A new GenAI Jailbreak Technique
Meet “Skeleton Key,” a newly discovered generative AI jailbreak technique that can bypass multiple AI models’ safeguards. Tested from April to May 2024, this multi-turn strategy tricks models into ignoring their responsible AI guardrails, allowing users to extract normally forbidden content. Microsoft found it effective against several major AI models, including those from Meta, Google, OpenAI, and Anthropic. The company has already implemented mitigations in their AI offerings and shared findings with other providers. They recommend various protective measures like input/output filtering, carefully crafted system messages, and abuse monitoring. This discovery underscores the ongoing challenge of securing AI systems as they become more powerful and widespread, highlighting the need for continuous vigilance and improvement in AI security practices.
Categorizing AI Risks
https://arxiv.org/pdf/2406.17864
Another interesting research paper titled “AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies” presents a comprehensive taxonomy of AI risks derived from government policies and corporate practices. The researchers analyzed 8 government policies from the EU, US, and China, along with 16 company policies from 9 leading AI firms to create a unified framework for AI risk assessment. Their taxonomy identifies 314 unique risk categories organized into a four-tiered structure, covering System & Operational Risks, Content Safety Risks, Societal Risks, and Legal & Rights Risks.
This work provides valuable insights into how different entities conceptualize and address AI risks. The researchers found that company policies often cover a broader range of risks than government regulations, highlighting potential gaps in enforcement. They also observed significant variations in risk categorization across companies and jurisdictions. This taxonomy aims to facilitate better communication and collaboration among policymakers, industry leaders, and researchers in addressing AI safety concerns. By offering a common language for discussing AI risks, the AIR 2024 taxonomy could prove instrumental in developing more effective policies and benchmarks for responsible AI development and deployment.
Have a Great Weekend!
You can chat with the newsletter archive at **https://chat.openai.com/g/g-IjiJNup7g-thinks-and-links-digest