open-source intelligence

OSINT sources can be divided up into six different categories of information flow:{{Cite book|title=The US Intelligence Community |isbn=978-0813349183 |last1=Richelson |first1=Jeffrey |year=2016 |publisher=Avalon }}

• Media: print newspapers, magazines, radio, and television from across and between countries.
• Internet: online publications, blogs, discussion groups, citizen media (i.e. – cell phone videos, and user created content), YouTube, and other social media websites (i.e. – Facebook, Twitter, Instagram, etc.). This source also outpaces a variety of other sources due to its timeliness and ease of access.
• Public government data: public government reports, budgets, hearings, telephone directories, press conferences, websites, and speeches. Although this source comes from an official source they are publicly accessible and may be used openly and freely.
• Professional and academic publications: information acquired from journals, conferences, symposia, academic papers, dissertations, and theses.
• Commercial data: commercial imagery, financial and industrial assessments, and databases.
• Grey literature: technical reports, preprints, patents, working papers, business documents, unpublished works, and newsletters.

OSINT is distinguished from research in that it applies the process of intelligence to create tailored knowledge supportive of a specific decision by a specific individual or group.{{cite web |title=Spy Agencies Turn to Newspapers, NPR, and Wikipedia for Information: The intelligence community is learning to value 'open-source' information |url=https://www.usnews.com/articles/news/national/2008/09/12/spy-agencies-turn-to-newspapers-npr-and-wikipedia-for-information.html |access-date=2008-09-15 |archive-url=https://web.archive.org/web/20121023193900/http://www.usnews.com/news/national/articles/2008/09/12/spy-agencies-turn-to-newspapers-npr-and-wikipedia-for-information |archive-date=2012-10-23 |url-status=live }}

Collecting open-source intelligence is achieved in a variety of different ways,{{Cite web |last=Leos |first=Devan |date=2023-02-28 |title=Thinking Like a Spy: How Open Source Intelligence Can Give You a Competitive Advantage |url=https://www.entrepreneur.com/growing-a-business/thinking-like-a-spy-how-open-source-intelligence-can-give/444634 |access-date=2023-11-08 |website=Entrepreneur |language=en}} such as:

• Social Media Intelligence, which is acquired from viewing or observing a subject's online social profile activity.
• Search engine data mining or scraping.
• Public records checking.
• Information matching and verification from data broker services.

OSINT, broadly defined, involves gathering and analyzing publicly accessible information to produce actionable insights.{{Cite web |title=What is OSINT (Open-Source Intelligence?) {{!}} SANS Institute |url=https://www.sans.org/blog/what-is-open-source-intelligence/ |access-date=2025-02-18 |website=SANS Institute}}

The U.S. Department of Homeland Security defines OSINT as intelligence derived from publicly available information, collected and disseminated promptly to address specific intelligence needs.[https://www.dhs.gov/sites/default/files/2022-09/Ethical%20Frameworks%20in%20OSINT%20Final.pdf Ethical Frameworks in Open Source Intelligence]. (Report) (2022). Part of the 2022 Public Private Partnership Analytic Exchange Program. Washington, DC: US Department of Homeland Security.

NATO describes OSINT as intelligence obtained from publicly available information and other unclassified data with limited public distribution or access.{{Cite book |url=https://archive.org/details/NATOOSINTHandbookV1.2/mode/2up |title=NATO OSINT Handbook V 1.2 |date=2001-11-01 |publisher=NATO}}

The European Union defines OSINT as the collecting and analyzing information from open sources to generate actionable intelligence, supporting areas like national security, law enforcement, and business intelligence.{{Cite web |date=2022-05-02 |title=OSINT: Open-source intelligence |url=https://data.europa.eu/en/publications/datastories/what-osint-open-source-intelligence |access-date=2025-02-18 |website=European Union Data}}

The United Nations has also recognized OSINT’s potential, noting its value in monitoring member states’ compliance with international regulations across various sectors, including public health and human rights.{{Cite web |last=Bochert |first=Florian |date=2021-11-19 |title=OSINT – The Untapped Treasure Trove of United Nations Organizations |url=https://hir.harvard.edu/osint-the-untapped-treasure-trove-of-united-nations-organizations/ |access-date=2025-02-18 |website=Harvard International Review |language=en}}

In the private sector, companies like IBM define OSINT as the process of gathering and analyzing publicly available information to assess threats, inform decisions, or answer specific questions. Similarly, cybersecurity firms such as CrowdStrike describe OSINT as the act of collecting and analyzing publicly available data for intelligence purposes.{{Cite web |last=Baker |first=Kurt |date=2025-01-17 |title=What is OSINT Open Source Intelligence? {{!}} CrowdStrike |url=https://www.crowdstrike.com/en-us/cybersecurity-101/threat-intelligence/open-source-intelligence-osint/ |access-date=2025-02-18 |website=Crowdstrike |language=en-US}}

File:911 commission seal.svg]]

OSINT practices have been documented as early as the mid-19th century in the United States and early 20th century in the United Kingdom.{{Cite journal |last=Block |first=Ludo |date=2023 |title=The long history of OSINT |journal=Journal of Intelligence History |volume=23 |issue=2 |pages=95–109 |language=en |doi=10.1080/16161262.2023.2224091 |issn=1616-1262|doi-access=free |hdl=1887/3731669 |hdl-access=free }}

OSINT in the United States traces its origins to the 1941 creation of the Foreign Broadcast Monitoring Service (FBMS), an agency responsible for the monitoring of foreign broadcasts. An example of their work was the correlation of changes in the price of oranges in Paris with successful bombings of railway bridges during World War II.{{cite web|last1=Bornn|first1=D Marshall|title=Service members, civilians learn to harness power of 'Open Source' information|url=https://www.army.mil/article/94007/Service_members__civilians_learn_to_harness_power_of__Open_Source__information|website=www.army.mil|access-date=14 May 2017|language=en|date=9 Jan 2013|archive-url=https://web.archive.org/web/20171209101817/https://www.army.mil/article/94007/Service_members__civilians_learn_to_harness_power_of__Open_Source__information|archive-date=9 December 2017|url-status=live}}

The Aspin-Brown Commission stated in 1996 that US access to open sources was "severely deficient" and that this should be a "top priority" for both funding and DCI attention.{{Cite book|title=The Five Disciplines of Intelligence Collection|last1=Lowenthal|first1=Mark|last2=Clark|first2=Robert|publisher=CQ Press|year=2015|isbn=978-1483381114|page=18}}

In July 2004, following the September 11 attacks, the 9/11 Commission recommended the creation of an open-source intelligence agency.See page 413 of the [http://www.9-11commission.gov/report/911Report_FM.pdf 9-11 Commission Report (pdf)] {{Webarchive|url=https://web.archive.org/web/20070705012031/http://www.9-11commission.gov/report/911Report_FM.pdf |date=2007-07-05 }}. In March 2005, the Iraq Intelligence Commission recommended{{Cite news|last=McLaughlin|first=Michael|date=June 2012|title=Using open source intelligence for cybersecurity intelligence|language=en-GB|work=ComputerWeekly.com|url=https://www.computerweekly.com/tip/Using-open-source-intelligence-software-for-cybersecurity-intelligence|url-status=live|access-date=2018-06-29|archive-url=https://web.archive.org/web/20180629155103/https://www.computerweekly.com/tip/Using-open-source-intelligence-software-for-cybersecurity-intelligence|archive-date=2018-06-29}} the creation of an open-source directorate at the CIA.

Following these recommendations, in November 2005 the Director of National Intelligence announced the creation of the DNI Open Source Center. The Center was established to collect information available from "the Internet, databases, press, radio, television, video, geospatial data, photos and commercial imagery."Office of the Director of National Intelligence. "[http://www.dni.gov/press_releases/20051108_release.htm ODNI Announces Establishment of Open Source Center] {{webarchive|url=https://web.archive.org/web/20060623072458/http://dni.gov/press_releases/20051108_release.htm |date=2006-06-23 }}". Press release, 8 November 2005. In addition to collecting openly available information, it would train analysts to make better use of this information. The center absorbed the CIA's previously existing Foreign Broadcast Information Service (FBIS), originally established in 1941, with FBIS head Douglas Naquin named as director of the center.Ensor, David. "[http://www.cnn.com/2005/POLITICS/11/08/sr.tues/ The Situation Report: Open source intelligence center] {{Webarchive|url=https://web.archive.org/web/20070325142701/http://www.cnn.com/2005/POLITICS/11/08/sr.tues/ |date=2007-03-25 }}". CNN, 8 November 2005. Then, following the events of 9/11 the Intelligence Reform and Terrorism Prevention Act merged FBIS and other research elements into the Office of the Director of National Intelligence creating the Open Source Enterprise.

Furthermore, the private sector has invested in tools which aid in OSINT collection and analysis. Specifically, In-Q-Tel, a Central Intelligence Agency supported venture capital firm in Arlington, VA assisted companies develop web-monitoring and predictive analysis tools.

In December 2005, the Director of National Intelligence appointed Eliot A. Jardines as the Assistant Deputy Director of National Intelligence for Open Source to serve as the Intelligence Community's senior intelligence officer for open source and to provide strategy, guidance and oversight for the National Open Source Enterprise.Office of the Director of National Intelligence "[http://www.dni.gov/press_releases/20051207_release.htm ODNI Senior Leadership Announcement] {{webarchive|url=https://web.archive.org/web/20060623072305/http://dni.gov/press_releases/20051207_release.htm |date=2006-06-23 }}". Press release, 7 December 2005. Mr. Jardines has established the National Open Source Enterprise[http://upload.wikimedia.org/wikipedia/en/b/b4/NationalOpenSourceEnterprise.pdf "National Open Source Entreprise Vision Statement"] {{webarchive|url=https://web.archive.org/web/20070928150640/http://upload.wikimedia.org/wikipedia/en/b/b4/NationalOpenSourceEnterprise.pdf |date=2007-09-28 }} May 2006 and authored intelligence community directive 301. In 2008, Mr. Jardines returned to the private sector and was succeeded by Dan Butler who is ADDNI/OS[http://www.dniopensource.org/Conference/Agenda.aspx DNI Open Source Conference 2008 "Decision Advantage" agenda, Office of the Director of National Intelligence, July 2008.] {{webarchive|url=https://web.archive.org/web/20100417020406/http://www.dniopensource.org/Conference/Agenda.aspx |date=2010-04-17 }} and previously Mr. Jardines' Senior Advisor for Policy.[http://www.dniopensource2007.com/sessions.cfm DNI Open Source Conference 2007 "Expanding the Horizons" agenda, Office of the Director of National Intelligence, July 2007.] {{webarchive|url=https://web.archive.org/web/20080801081412/http://www.dniopensource2007.com/sessions.cfm |date=2008-08-01 }}

alt=Systematic Software Engineering uses open source intelligence, compiled by [[Janes Information Services, as part of its intelligence platform.|thumb|Open source intelligence may be ingested to battle management systems such as CPCE by Systematic, which uses an open source feed from Janes Information Services.]]

Open-source intelligence (OSINT) relies on a wide range of tools and platforms to collect, analyze, and validate publicly available information. These tools vary from general-purpose web browsers to specialized software and frameworks designed specifically for open-source investigations.

The web browser serves as a foundational tool in OSINT workflows, granting access to vast amounts of publicly available data across websites, forums, blogs, and databases. It also enables the use of both open-source and proprietary software tools—either purpose-built for OSINT or adaptable for intelligence-gathering purposes.

A number of dedicated tools and platforms have been developed to streamline the process of gathering and analyzing open-source information. These include:

• Maltego : A data visualization tool used for link analysis and reconnaissance.
• Shodan : A search engine for internet-connected devices, often used in cybersecurity research.
• TheHarvester : A tool used for e-mail harvesting and subdomain enumeration.
• FOCA (Fingerprinting Organizations with Collected Archives) : Used to extract metadata and find hidden information in documents.
• Recon-ng : A full-featured web reconnaissance framework written in Python.

The OSINT Framework , an open-source project maintained on GitHub, provides a categorized directory of over 30 major types of OSINT tools, covering areas such as social media investigation, geolocation, domain analysis, and more.[3]

Given the evolving nature of digital platforms and online tools, continuous learning is essential for effective OSINT practice. Numerous educational organizations, investigative groups, and training institutions offer resources to support skill development in this field. Notable contributors include:

• Bellingcat : An independent collective known for its open-source investigative journalism.
• SANS Institute : Offers courses and certifications focused on cybersecurity and digital forensics, including OSINT-related topics.
• IntelTechniques : Provides training materials and tools for journalists, law enforcement, and researchers.

Books such as Open Source Intelligence Techniques by Michael Bazzell serve as practical guides to navigating the digital landscape, offering curated lists of tools and techniques across multiple domains. However, due to the rapid pace of change in the online environment, the author emphasizes the importance of ongoing study, training, and adaptation to maintain relevance and effectiveness in OSINT operations.[1]

As OSINT practitioners often conduct sensitive or public investigations, maintaining personal safety and operational security is critical. Analysts may employ various tools to protect their identity and digital footprint. Ryan Fedasiuk, an analyst at the Center for Security and Emerging Technology, recommends several OPSEC best practices and tools, including:

• Virtual Private Networks (VPNs)
• Cached webpage services (e.g., Google Cache, Wayback Machine)
• Digital archive services
• URL and file scanners (e.g., VirusTotal)
• Browser sandbox applications
• Antivirus software

These tools help reduce exposure to potential threats when conducting online investigations, especially when researching adversarial or high-risk subjects.[2]

In institutional settings, OSINT is often integrated into broader command and control systems. For example, CPCE (Command Post Communications Environment) by Systematic incorporates open-source feeds such as those provided by Jane’s Information Services , enabling real-time intelligence integration into military and defense operations.

One of the primary challenges in open-source intelligence (OSINT) is the sheer volume of publicly available information, often referred to as the "information explosion." The exponential growth of digital content across news platforms, social media, forums, blogs, and official publications presents significant difficulties for analysts attempting to identify, verify, and contextualize relevant data.

The rapid pace at which new information is generated often outstrips the capacity of analysts to process and evaluate it effectively. This can lead to difficulties in distinguishing reliable sources from misinformation or disinformation, and in prioritizing intelligence that is both timely and actionable.

To mitigate these challenges, some organizations have explored the use of automated tools, machine learning algorithms, and crowdsourcing techniques. While large-scale automation remains a developing field, limited efforts involving amateur or citizen analysts have occasionally contributed to the filtering and categorization of open-source data—though such methods are generally considered supplementary rather than definitive.

While OSINT involves only legally accessible, publicly available information, the distinction between lawful research and illegal activity becomes critical when individuals or entities misuse open-source practices.

In most jurisdictions, the unauthorized collection and transmission of sensitive information to a foreign government or intelligence agency—even if obtained from public sources—can constitute espionage , particularly if it involves strategic, military, or national security-related data. Espionage of this nature, distinct from treason (which typically involves betrayal of one’s own state), has historically been employed by states as a tool of diplomacy, warfare, and influence.

It is important to note that while OSINT itself is a legitimate and transparent discipline, its misuse—particularly when combined with covert intent or malicious purpose—can cross into legally and ethically prohibited territory. Therefore, responsible OSINT practitioners emphasize adherence to legal standards, ethical guidelines, and operational security best practices.

As open-source intelligence (OSINT) has grown in prominence across government, military, corporate, and journalistic sectors, a number of professional associations and certification programs have emerged to support practitioners, standardize methodologies, and promote ethical conduct.

The OSINT Foundation is a U.S.-based professional association dedicated to advancing the practice of open-source intelligence within the U.S. Intelligence Community and beyond. Open exclusively to U.S. citizens, the organization aims to elevate the visibility and professionalism of OSINT as a formal intelligence discipline. It serves as a platform for knowledge sharing, networking, and advocacy among current and aspiring OSINT practitioners.

OSMOSIS , an offshoot of the Hetherington Group—a private investigation and corporate intelligence firm—offers training programs and conferences that lead to the Open-Source Certified (OSC) designation. The OSC program was developed to help formalize and standardize professional OSINT practices.

According to the program’s guidelines, candidates must fulfill specific prerequisites and pass a 100-question examination to demonstrate proficiency in open-source research and analysis. The certification emphasizes legal compliance, ethical behavior, and technical competence in gathering and interpreting publicly available information.

IntelTechniques , a provider of investigative and OSINT training, offers the Open Source Intelligence Professional (OSIP) certification. The program is designed to test participants' ability to produce actionable intelligence using real-world scenarios and standardized evaluation criteria.

In addition to structured courses, IntelTechniques fosters a moderated online community where professionals can exchange insights on best practices, tools, and methodologies. While the OSIP certification is optional, participation in the training itself helps individuals develop and refine their OSINT skills for use in law enforcement, journalism, cybersecurity, and corporate investigations.

Beyond formal certifications, several organizations offer specialized OSINT training and platforms for professional development:

• Bellingcat, an independent collective known for its open-source investigative journalism, provides public training sessions and resources for journalists, researchers, and activists.
• Independent educators and trainers also contribute to the field through blogs, podcasts, YouTube channels, and social media communities focused on OSINT tradecraft.

These informal and semi-formal learning environments play a vital role in expanding access to OSINT education and fostering global collaboration among practitioners.

{{div col|colwidth=20em}}

• {{annotated link|Ashley Feinberg}}
• {{annotated link|Bellingcat}}
• {{annotated link|Co-occurrence networks}}
• {{annotated link|Dan Butler (civil servant)}}
• {{annotated link|DARPA TIDES program}}
• {{annotated link|Doxing}}
• {{annotated link|Eliot A. Jardines}}
• {{annotated link|Eliot Higgins}}
• {{annotated link|Fusion center}}
• {{annotated link|ICWatch}}
• {{annotated link|Intellipedia}}
• {{annotated link|Investigative Data Warehouse}}
• {{annotated link|MiTAP}}
• {{annotated link|National Intelligence Open Source Committee}}
• {{annotated link|NATO Open Source Intelligence Handbook}}, NATO Open Source Intelligence Reader
• {{annotated link|Open data}}
• {{annotated link|Open Source Center}}
• {{annotated link|Oryx (blog)}}
• {{annotated link|Private intelligence agency}}
• {{annotated link|Social cloud computing}}
• {{annotated link|Special Libraries Association}}
• {{annotated link|Strategic intelligence}}
• {{annotated link|Open-source intelligence in the 2022 Russian invasion of Ukraine}}

{{Div col end}}

{{Reflist|2}}

• [http://www.washtimes.com/national/20060418-110124-3694r.htm WashTimes.com], Washington Times – CIA mines 'rich' content from blogs, 19 April 2006
• [https://web.archive.org/web/20081201184220/http://www.gcn.com/print/25_6/40152-1.html GCN.com], Government Computer News – Intelligence units mine the benefits of public sources 20 March 2006
• [https://web.archive.org/web/20080310024429/http://findarticles.com/p/articles/mi_m0IBS/is_4_31/ai_n16419797 FindAcricles.com], Military Intelligence Professional Bulletin October–December, 2005 by Barbara G. Fast
• [https://fas.org/irp/congress/2005_hr/062105jardines.pdf FAS.org], Congressional Testimony on OSINT and Homeland Security 21 June 2005
• [https://www.forbes.com/2008/11/21/maltego-data-mining-identity08-tech-cz-tb_1121maltego.html Forbes.com], When Everyone Can Mine Your Data by Taylor Buley, 11.21.08]
• {{cite web | last=Thompson | first=Clive | title=Open-Source Spying | website=The New York Times | date=2006-12-03 | url=https://www.nytimes.com/2006/12/03/magazine/03intelligence.html | access-date=2018-05-29}}
• {{cite web | title=Open Source Intelligence (OSINT) | website=RIS Open Source Intelligence | date=2018-05-29 | url=http://arnoreuser.com/ | ref={{sfnref | RIS Open Source Intelligence | 2018}} | access-date=2018-05-29}}
• {{cite web | title=The Intelligence Network : I n t r o d u c t i o n | website=intellnet.org | date=2008-05-09 | url=http://www.intellnet.org/ | archive-url=https://web.archive.org/web/20080509085418/http://www.intellnet.org/ | archive-date=2008-05-09 | url-status=unfit | ref={{sfnref | intellnet.org | 2008}} | access-date=2018-05-29}}

• {{cite book |author=United Nations Office of the High Commissioner for Human Rights |author-link=United Nations Office of the High Commissioner for Human Rights |author2=University of California, Berkeley Human Rights Center |author2-link=University Human Rights Centers#Human Rights Center – University of California, Berkeley |title=Berkeley Protocol on Digital Open Source Investigations A Practical Guide on the Effective Use of Digital Open Source Information in Investigating Violations of International Criminal, Human Rights and Humanitarian Law |url=https://www.ohchr.org/sites/default/files/2024-01/OHCHR_BerkeleyProtocol.pdf |publication-place=New York, Geneva |publisher=UN Office of the High Commissioner for Human Rights; Human Rights Center, UC Berkeley School of Law |year=2022 |isbn=978-92-1-154233-2 |oclc=1334608062}}

• Deneuville, A., Hernández López, G. & Rasmi, J. (Eds.) 'Contre-enquêtes en sources ouvertes'. Multitudes, 89, 2022.
• Arthur S. Hulnick: '[https://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780195375886.001.0001/oxfordhb-9780195375886-e-0014 The Dilemma of Open Source Intelligence: Is OSINT Really Intelligence?]', pages 229–241, The Oxford Handbook of National Security Intelligence, 2010
• [http://epublications.bond.edu.au/cgi/viewcontent.cgi?article=1010&context=cewces_papers Cody Burke: 'Freeing knowledge, telling secrets: Open source intelligence and development', Bond University, May 2007] {{Webarchive|url=https://web.archive.org/web/20160817134042/http://epublications.bond.edu.au/cgi/viewcontent.cgi?article=1010&context=cewces_papers |date=2016-08-17 }}
• [http://www.isn.ethz.ch/isn/Digital-Library/Publications/Detail/?id=122008 Florian Schaurer, Jan Störger: 'The Evolution of Open Source Intelligence', OSINT Report 3/2010, ISN, ETH Zürich, October 2010]
• {{Cite book|last=Hassan, Nihad A.|url=https://www.worldcat.org/oclc/1043830928|title=Open source intelligence methods and tools : a practical guide to online intelligence|others=Hijazi, Rami|date=30 June 2018|isbn=978-1-4842-3213-2|location=[United States]|oclc=1043830928}}

• [http://rr.reuser.biz The Open Source Intelligence Resource Discovery Toolkit]
• [https://web.archive.org/web/20020307050442/http://www.time.com/time/covers/1101020311/viewpoint.html The New Craft of Intelligence: Making the Most of Open Private Sector Knowledge]
• [https://web.archive.org/web/20080216224135/http://www.cm2limited.com/casestudies/casestudies.php Actual Intelligence Case Studies Leveraging Open Source Intelligence (OSINT)]
• [https://web.archive.org/web/20070613113235/https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/csi-studies/studies/vol48no3/article05.html Sailing the Sea of OSINT in the Information Age]
• [https://fas.org/sgp/crs/intel/RL34270.pdf Open Source Intelligence (OSINT): Issues for Congress], Congressional Research Service, December 5, 2007
• [https://web.archive.org/web/20160304031047/http://www.osint.org/crs-report-osint.pdf Open Source Intelligence (OSINT): Issues for Congress], Congressional Research Service, January 28, 2008
• [https://www.thefreelibrary.com/FMSO-JRIC+and+Open+Source+Intelligence%3A+speaking+prose+in+a+world+of...-a0146354022 The Free Library], FMSO-JRIC and Open Source Intelligence: speaking prose in a world of verse, Military Intelligence Professional Bulletin, Oct–Dec, 2005 by Jacob W. Kipp

{{Intelligence cycle management}}

{{Authority control}}

{{DEFAULTSORT:Open Source Intelligence}}

Category:Applied data mining

Category:Intelligence gathering disciplines

Category:Collective intelligence

Category:American inventions

Category:2005 introductions

Category:2005 establishments in the United States

Category:Management cybernetics