Thinks and Links | May 5, 2024
BD&A - Thinks and Links
Big Data & Analytics - Thinks and Links | News and insights at the intersection of cybersecurity, data, and AI
đź“« Subscribe to Thinks & Links direct to your inbox
Happy Weekend!
AI Executive Order - 180 Days Later
The Biden Administration published the Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence on October 30, 2023. Much of the content of the order required various federal agencies to publish their own AI guidance, regulations, and rules within 180 days. That deadline was Saturday, April 27. So, over the last few weeks we’ve seen an explosion of rulemaking and guidance for AI in many different domains.
A few of the most notable are shared and summarized below, but the TLDR is that AI guidance, use cases, and rules are plentiful. When organizations started to really invest in Generative AI last summer, it was as if there were too few guidelines for how to do this safe and secure. Now it can seem like there are too many guidelines. The core of AI protection in the US seems to run through NIST. Most of the guidance that is provided directly references the NIST AI Risk Management Framework and fully aligns to those guidelines. If you only read one US Federal Government website on AI, make it the NIST AI RMF - of course while you’re there you’ll want to read the original RMF, the Playbook of suggested ways to implement and align, and then the various publications specific to Generative AI and recent developments.
Here are a few more reports and developments since the AI EO worth reviewing:
Department of Homeland Security
Announcement: https://www.dhs.gov/news/2024/04/29/dhs-publishes-guidelines-and-report-secure-critical-infrastructure-and-weapons-mass
The guideline summarizes and aligns broad AI Security topics into the 16 critical infrastructure sectors. It discusses three main categories of AI risk: attacks using AI, attacks targeting AI systems, and failures in AI design and implementation. They then provide recommendations across the four NIST AI RMF functions: Govern, Map, Measure, and Manage. It provides guidance for critical infrastructure owners and operators to establish an organizational culture of AI risk management, understand their individual AI use context and risk profile, develop systems to assess and track AI risks, and prioritize actions to manage safety and security risks.
More details at https://www.dhs.gov/ai and https://www.dhs.gov/publication/ai-roadmap
White House Office of Science and Technology Policy (OSTP)
Announcement: https://www.whitehouse.gov/ostp/news-updates/2024/04/29/framework-for-nucleic-acid-synthesis-screening/
The White House Office of Science and Technology Policy (OSTP) has released a new framework to enhance the screening of synthetic nucleic acid purchases and improve biosecurity measures. This framework, mandated in the AI Executive Order is meant to increase controls around the use of AI for synthesis and development of potential bioweapons or other DNA / RNA based threats.
Department of Commerce | National Insititute of Standards and Technology (NIST)
Announcement: https://www.commerce.gov/news/press-releases/2024/04/department-commerce-announces-new-actions-implement-president-bidens
Several initiatives fall under Commerce, including updates to the NIST AI RMF to incorporate Generative AI specific guidance and safeguarding data used for training AI systems and launching a competition on AI content detection.
Department of Energy
Announcement: https://www.energy.gov/articles/doe-announces-new-actions-enhance-americas-global-leadership-artificial-intelligence
The Department of Energy released a report on opportunities to use AI for acceleration of climate change goes including grid planning, permitting and siting, grid operations and reliability, and grid resilience. This also includes the creation of “PolicyAI” a LLM-based tool that can help the department and stakeholders search through over 50 years of documents related to the environment, development, and agency actions. They also published their Risk and Benefit review of AI for Critical AI Infrastructure along with many more resources and use cases for AI.
Department of Treasury
Announcement: https://home.treasury.gov/news/press-releases/jy2212
The Treasury released guidance ahead of schedule in late March, with details on managing AI-specific cybersecurity risks in the financial services sector. The guidance highlights the increasing adoption of AI systems for cybersecurity and fraud detection by financial institutions. The report discusses data poisoning, leakage, and integrity attacks; adversarial AI attacks exploiting system vulnerabilities; and risks from reliance on third-party AI providers. Treasury emphasizes best practices such as situating AI risk management within existing enterprise risk frameworks, mapping data supply chains, implementing secure-by-design principles, and collaborating across the sector to share information and develop standards.
President’s Council of Advisors on Science and Technology (PCAST)
As requested in the Executive Order, this report elaborates on the opportunities for I to accelerate research across many fields by making more efficient discovery of solutions, automating routine tasks, enhancing simulations, and democratizing access to information. The recommendations of the report include encouraging increased use of AI for cross-domain collaboration, encouraging the use of transparent, responsible, and trustworthy AI solutions in scientific work, and adopting principles of responsible AI throughout the research process.
Department of Labor
Announcement: https://www.dol.gov/agencies/ofccp/ai/ai-eeo-guide
Guidelines for federal contractors using AI including the importance of protecting against bias and discrimination, staying compliant with equal employment opportunity obligations, and not overly relying on AI for performing tasks.
Health and Human Services
This document provides guidelines for State, Local, Tribal, and Territorial Governments for working with AI. It includes details about classifying AI by risk - similar to how the EU’s AI Act approaches regulatory oversight. It discusses policy recommendations around responsible AI innovation, risk management, options to “opt out” of AI, human oversight, and strengthening AI governance.
National Security Agency
The NSA also provides excellent guidance for deployment and control of AI, especially when it was designed by another organization (e.g. could have vulnerabilities). These recomendations include:
Secure the deployment environment by managing governance, ensuring a robust architecture, and hardening configurations.
Continuously protect the AI system by validating it before and during use, securing exposed APIs, actively monitoring model behavior, and protecting model weights.
Secure AI operation and maintenance by enforcing strict access controls, ensuring user awareness and training, conducting audits and penetration testing, implementing robust logging and monitoring, updating and patching regularly, preparing for high availability and disaster recovery, and planning secure delete capabilities.
The NSA emphasizes that AI systems are software systems and should follow secure by design principles. Particular importance is placed on measures like compromise assessments, IT environment hardening, supply chain security reviews, access controls, logging and monitoring, and protecting model weights. Ongoing risk identification, mitigation implementation, and issue monitoring are also shared as essential for securing AI systems.
Housing and Urban Development
Announcement: https://www.hud.gov/press/press_releases_media_advisories/HUD_No_24_098
Two guidance documents were released on Thursday, addressing the application of the Fair Housing Act to the use of artificial intelligence in housing-related practices. The first focuses on tenant screening, providing best practices for housing providers and screening companies to ensure fair, transparent, and non-discriminatory policies when using AI and algorithms. The second addresses the use of targeted advertising on online platforms, cautioning advertisers and platforms about the risks of violating the Fair Housing Act by denying consumers information about housing opportunities based on protected characteristics.
Office of Personnel Management
Announcement: https://www.opm.gov/data/resources/ai-guidance/
The Federal Government’s AI Guidance for employees was mandated by the Executive Order, but also serves as a great template for guidance to workers who are embracing Generative AI’s potential while staying aware of the risks.
You can also track how the Federal Government’s surge in AI talent is going thanks to another mandate from the order. Additional reporting and reviews are also occuring to ensure the appropriate levels of talent to support the many initiatives across the government.
A Beautiful Mess that Nobody Needs
https://futurism.com/the-byte/pricey-ai-device-android-app-extra-steps
Wearable AI devices launched over the past few months claim to be life changing experiences. The initial reviews are in and they are… underwhelming. The rush to capitalize on the AI Hype has necessarily caused companies to move fast. “Secure the bag” while the funding is good. And the vision of wearable devices that can act as life assistants is alluring, but requires a lot of hardware engineering to compliment the AI operating systems. While Nvidia is making incredible strides at crossing the digital / real world innovation gap - the truth is that today it takes time to build the hardware that is worthy of the AI hype. Today’s sticker shock is really a down payment to build more useful apps in the future. None of this should be surprising, but the reviews are amusing
DrEureka closes the simulation-reality gap
https://eureka-research.github.io/dr-eureka/
This research paper from Nvidia, UPenn, and UT Austin showcases the rapidity with which a robot dog can be taught to balance on a bouncy ball - a task chosen because it is very hard to simulate - and the speed with which the capability was learned in a simulation and transferred over to hardware. This demonstrates an impressive use of LLMs to design the operating systems for robotics and points to one more real-world use case of AI that needs to be secured. Who knows what else the AI can be training these robots to do! A backdoor or missed edge case could lead to hardware running AI-developed software that could be later exploited. A scary, but increasingly real concern operators of these tools must begin to grapple with.
Free AI Security Bots
https://github.com/openai/openai-security-bots
OpenAI has published several example Slackbots that can help to automate security functions. The tools are starting points for developing custom incident response, triaging tickets, and supporting SDLC. This will interact with the public OpenAI API, so if you have requirements to use a different LLM, modification will be needed. Even if you don’t use the scripts, there are some great prompts within the code, such as: “You’re a highly skilled security analyst who is excellent at asking the right questions to determine the true risk of a development project to your organization. You work at a small company with a small security team with limited resources. You ruthlessly prioritize your team’s time to ensure that you can reduce the greatest amount of security risk with your limited resources. Please provide a summary of the key security design elements, potential vulnerabilities, and recommended mitigation strategies presented in the following project document. Highlight any areas of particular concern and emphasize best practices that have been implemented. Also outline all key technical aspects of the project that you assess would require a security review. Anything that deals with data, end users, authentication, authorization, encryption, untrusted user input, internet exposure, new features or risky technologies like file processing, xml parsing and so on”* {{moved}}
Bobby Tables but with LLM Apps - Google NotebookLM Data Exfiltration
https://embracethered.com/blog/posts/2024/google-notebook-ml-data-exfiltration/
A walkthrough of how user provided data can include prompt injection and data exfiltration code. The example provided was responsibly disclosed and eventually patched. But this exploit is relatively simple and can be easily used to inject malicious/false data, delete data, or otherwise manipulate the rest of the application with a few lines of prompt injection. While chat interfaces might be monitored for unusual / harmful prompts, this example included the prompt as a part of the data being uploaded. As applications that use generative AI become more widespread, we’ll see more of these kinds of exploits.
Have a Great Weekend!
May the Fourth Be With You
You can chat with the newsletter archive at **https://chat.openai.com/g/g-IjiJNup7g-thinks-and-links-digest