We’re going to step back in time to when the World Wide Web didn’t exist, as it’s possible that we can learn things about how to do OSINT investigations from the past that remain relevant today.
Although the internet now provides instant access to a stunning diversity of resources from wherever a connection to it can be found, researchers may be wise to look to a time before this facility came to be. In the past, I’ve enjoyed access to treasure troves of data from national and state libraries and archives, university libraries, private organizational libraries, and local municipal libraries. Before the dawn of the public internet, they contained mountains of information useful to open-source investigators, often data you would be hard-pressed to find elsewhere.
And they still do.
Old tools such as physical telephone books and city directories have been replaced by people search services, search engines and other tools today. And these current means will be replaced by other tools in the future. This is another reason OSINT researchers should focus on their investigative and analytical processes, not tools; tools and who controls them inevitably change over time and are eventually replaced by others. But aggregated collection points of information, like a public library, will never go out of style.
When the World Wide Web came into being and as it expanded, libraries actually gained resources! Now they could be used to access limited-availability online databases, provide their own resources remotely, and continue to serve as key portals of information.
Many libraries have access to expensive databases or periodicals that they can provide to library users free of charge. Depending on the library, this may include access to databases such as Lexis/Nexis or periodical-style resources like Jane’s Online and other specialized publications such as medical journals. Such materials, both before and after the explosion of internet materials, were often otherwise difficult to access due to distribution limitations or cost.
Sometimes access to some of these resources are available to online library users, although in many cases a visit the library is necessary to use them. But it may be well worth a trip to the library for the OSINT researcher in many situations!
Larger library systems may include on their websites what resources are available. Those conducting OSINT research can plan ahead to find and record these pages for their local and national government and nearby university libraries, at the least. Some examples:
• Philadelphia Free Library
• US Library of Congress
• University of California, Santa Barbara Library
• National Library of Australia
Specialized libraries provide access to resources many are unaware of. For example, there are a great amount of government documents in the United States available through the Freedom of Information Act (FOIA). Other countries with similar open record laws often do the same. Government agencies regularly operate libraries where released records can be found. Private organizations that participate in this process may also share this information with the public through their own libraries as well.
Some popular US FOIA document resources include:
• US Federal Bureau of Investigation FOIA Library
• The Black Vault
• National Security Archive
There are other government resources have been available for decades but are now publicly available online, such as the Social Security Death Index (sometimes called the SSDI). There are many ways to access and search this index online including on genealogy sites such as ancestry.com. The SSDI has been useful to me as a US-based investigator on several occasions, if only to confirm that the passing of an individual is the reason there were no current leads on their whereabouts. Some other countries, states, and provinces have similar resources, although they may not contain records as recent. Resources such as these may not often come into play in any given investigation, but remain useful.
Researchers should keep in mind that for just about every conceivable area of interest – you can probably major libraries focused on it. Most of us are aware that these exist in some form, knowing that there are law libraries, for example. Many of these are associated with a government agency or educational institution.
There are so many different kinds of these specialized libraries around the world, it almost boggles the mind. These libraries inevitably have access to specialized datasets relevant to that specific field they address, which contain precisely the kind of information you are interested in. Often accessing the information requires a login to their website, or a visit.
US National Agriculture Library
Medicine? Of course.
US National Library of Medicine
TextielMuseum (The Netherlands)
Aerospace history? That too.
Smithsonian National Air and Space Museum Library
The above are single links provided as examples, but of course there are numerous libraries in many cases for a particular subject area, in various locales. The examples here are mostly based in the US, but many places have their own versions of many of these libraries, e.g. the UK’s National Aerospace Library or the Tohoku University Agricultural Library.
Corporate and organizational libraries are an oft over-looked resource. Most large organizations, both for- and non-profit, typically have a library. It may store little more than corporate records. However, these may also contain information not only on the enterprise, its personnel (past and present) and its activities, but even on its particular industry vertical including competitors and scientific information, access to very specific technical resources and databases, and more.
Do you want some information on road changes in a German village over time? The German Automobile Club (ADAC) library may be able to help you out. Investigating the status and activities of a charity in the US? Guidestar and The Foundation Center have tracked them for almost a century. Interested for some reason in The Linnean Society of London? Check out their library, which aside from the scientific material you would expect, has much information concerning their own organization. Trying to map connections between key executives years ago at a for-profit corporation, or find out about their product development process? They might provide access to their corporate library if you ask nicely, although you’ll probably need to come up with a nice-sounding reason as well to gain that access.
Which brings us to this: it’s worth remembering that open-source information you seek is not always available online and there are other methods of obtaining public information than opening up a web browser. As a private investigator, I sometimes needed public record data from a distant municipality which did not host the data on the web. Often a phone call to the clerk’s office or responsible administrator would provide me with the needed information. Some resources charge a fee or registration for online data, but it can be gotten for free by contacting them in other ways, sometimes even a phone call.
If we take this step of direct engagement, we add to our potential sources of data. A professional in a specific field, a witness to an incident who can provide more details than the local media gathered, or that official with access to the record you need may only be a phone call or an email away from providing you with valuable information that simply isn’t available otherwise.
Some researchers, such as journalists, do this often enough to keep it in mind, but many others may not think to take this step. In some cases, laws, security concerns or other restrictions may preclude directly contacting potential sources of information, but if permissible this should remain an option for any investigator. Is this purely OSINT? You can decide that for yourself – but it is assuredly a method that can move investigations forward.
4 thoughts on “Old Skool OSINT”
Great post. It is worth noting that the Social Security Death Index ceased providing new information to the public in 2014
The link to “Tokyo University Agricultural Library.” was “Tohoku University Agricultural Library.”
“Tohoku”, not “Tokyo”
Excellent catch B-SON! Thank you. We have updated the text of the blog. Really appreciate your attention to detail.
+1 – add MuckRock (not to be confused with MuckRack which is also good for other reasons) re: FOIA – https://www.muckrock.com/
This is a timeless piece; this is one of those lessons that needs to be taught and re-taught — and — for those of us who are busy and think we know these lessons already…we need to remain humble and remember the importance of “shoe leather” and the analog. My first voyages into the cyber realm of research go back as far as the first home semi-affordable modems in the 80s, but there was a book in 1992 called THE INFORMATION BROKER’S HANDBOOK by Sue Rugge and Alfred Glossbrenner (it can be found at archive.org) that simultaneously turned me on to the power, pros and cons (and cost) of aggregation (today, we call these services Aggregators) as well as the privacy implications. So, I simultaneously dove into digital OSINT, hacking — and art (a different story). And — early on, I joined the EFF because I could see where this was all headed.
I’m also really glad you brought up libraries. I’m amazed how often they are overlooked — even by students (undergrad and graduate leve). It’s insane. I’m writing a piece that might complement this, about the power of scholarly databases (especially Scopus, Web of Science and Dimensions). They hold a lot of power when it comes to social network analysis (SNA) and other forms of person and corporate intelligence. These databases are often overlooked because the average researcher (in my experience) is simply not aware of them — partly due to paywalls, but also due to the fact that research skills are not taught well in primary and secondary institutions (at least, not in the USA). But the librarians at the public and private libraries you refer to _are_ highly skilled and aware — all you need to do is ask. So, my advice, based on your article, is for every investigator and researcher out there to be friendly to every librarian you meet — and support the ALA.org if you can afford to.The profession is far from “dead” — it’s just in a state of constant reinvention. You’d be surprised what they can turn you on to.
Thanks for an inspiring read.
Comments are closed.