How Do We Know a Sequence Screening Program Is Effective?

With biosecurity screening, a no-news-is-good-news approach is not enough. 

Implementing a biosecurity program will likely relieve many of your most pressing concerns related to your ability to keep deadly pathogens out of the hands of those with malicious intent. Yet, once you’ve allayed those initial concerns, there will likely be new concerns to keep you up at night: Is your screening system that good or are you just that lucky? Is your system flagging the sequences it needs to, or are some sequences slipping by undetected? Is your system returning false positives because your parameters are too restrictive or because you’ve cast too wide a net? When dealing with deadly pathogens, it’s essential to know the answers.

One way to find the answers to these questions is to wait until you can look back from the future to see how well you did in the past. That approach might work with some things, but a more proactive approach is required when working with potential substances of concern. Since waiting it out and hoping for the best are both also unsatisfactory, those researchers and others involved in synthetic biology are exploring several options for evaluating their programs.

Testing Biosecurity Screening Program Effectiveness

It’s a certainty that you don’t want the only people looking for weaknesses in your biosecurity screening program to be those with ill intent. Because of this, the ability to assess the effectiveness of a screening program includes not just flagging a substance or sequence but also noting whether or not the appropriate action is taken when the flag occurs. This type of review can fall under the auspices of individual companies, industry groups, or government oversight committees, among others. Just how is it done?

1. Red teaming.  Many companies providing toxins, pathogens, and sequences have taken their cue from cybersecurity and engage in red teaming to test their screening programs by staging a deliberate attempt to defeat the screening systems. Those involved in the exercise purposely submit orders with deadly substances or substances with the known potential to be used to create deadly substances. If the sequence or substance is properly flagged, that provides information about the first level of substance screening effectiveness. If the information provided with the flag results in your company properly identifying the threat as valid, that provides the second layer of information. If appropriate action is taken as a result of the first two screening levels, you have an additional piece of essential information. With these data points, you can assign a grade or score to your screening effectiveness.

2. IGSC screening adherence certification. The International Gene Synthesis Consortium (IGSC) is an industry-led organization that, along with promoting the beneficial use of gene synthesis, is committed to safeguarding synthetic biology and applying a common protocol for screening DNA sequences and customers. Not all IGSC members synthesize DNA. Some are academic members that work on biosecurity or work with synthetic DNA. Others, like Battelle, develop biosecurity solutions. Those that do synthesize DNA must pass an initial test to show they can successfully screen DNA sequences. They must also agree to adhere to the IGSC Harmonized Screening Protocol v 2.0. 

3. ISO certification. The International Organization for Standardization (ISO) is an international organization with 164 national standards bodies as members. The ISO has developed over 23,000 international standards for all aspects of technology, manufacturing, and management. ISO 35001: Biorisk management for laboratories and other related organizations is the first standard to harmonize and deliver international best practices for effective biorisk management systems. This certification indicates that the biosecurity screening program is effective. 

The growth of dual-use research makes it ever-more essential that the biosecurity screening programs are not only in place but also are as effective as they purport to be. Stakeholders from academic institutions and governments to research labs and the companies that fill the orders for the substances and sequences necessary for their work are invested in doing all that is possible to ensure that their screening efforts are effective.