{"id":45093,"date":"2023-10-20T15:41:05","date_gmt":"2023-10-20T15:41:05","guid":{"rendered":"http:\/\/startupsmart.test\/2023\/10\/20\/could-this-tool-for-the-dark-web-fight-human-trafficking-and-worse-startupsmart\/"},"modified":"2023-10-20T15:41:05","modified_gmt":"2023-10-20T15:41:05","slug":"could-this-tool-for-the-dark-web-fight-human-trafficking-and-worse-startupsmart","status":"publish","type":"post","link":"https:\/\/www.startupsmart.com.au\/uncategorized\/could-this-tool-for-the-dark-web-fight-human-trafficking-and-worse-startupsmart\/","title":{"rendered":"Could this tool for the dark web fight human trafficking and\u00a0worse? – StartupSmart"},"content":{"rendered":"
In today\u2019s data-rich world, companies, governments and individuals want to analyse anything and everything they can get their hands on \u2013 and the World Wide Web has loads of information.<\/p>\n
At present, the most easily indexed material from the web is text.<\/p>\n
But as much as 89 to 96 percent of the content on the internet is actually something else \u2013 images, video, audio, in all thousands of different kinds of non-textual data types.<\/p>\n
Further, the vast majority of online content isn\u2019t available in a form that\u2019s easily indexed by electronic archiving systems like Google\u2019s.<\/p>\n
Rather, it requires a user to log in, or it is provided dynamically by a program running when a user visits the page.<\/p>\n
If we\u2019re going to catalog online human knowledge, we need to be sure we can get to and recognise all of it, and that we can do so automatically.<\/p>\n
How can we teach computers to recognise, index and search all the different types of material that\u2019s available online?<\/p>\n
Thanks to federal efforts in the global fight against human trafficking and weapons dealing, my research forms the basis for a new tool that can help with this effort.<\/p>\n
The \u201cdeep web\u201d and the \u201cdark web\u201d are often discussed in the context of scary news or films like \u201cDeep Web,\u201d in which young and intelligent criminals are getting away with illicit activities such as drug dealing and human trafficking \u2013 or even worse.<\/p>\n
But what do these terms mean?<\/p>\n
The \u201cdeep web\u201d has existed ever since businesses and organisations, including universities, put large databases online in ways people could not directly view.<\/p>\n
Rather than allowing anyone to get students\u2019 phone numbers and email addresses, for example, many universities require people to log in as members of the campus community before searching online directories for contact information.<\/p>\n
Online services such as Dropbox and Gmail are publicly accessible and part of the World Wide Web \u2013 but indexing a user\u2019s files and emails on these sites does require an individual login, which our project does not get involved with.<\/p>\n
The \u201csurface web\u201d is the online world we can see \u2013 shopping sites, businesses\u2019 information pages, news organisations and so on.<\/p>\n
The \u201cdeep web\u201d is closely related, but less visible, to human users and \u2013 in some ways more importantly \u2013 to search engines exploring the web to catalog it.<\/p>\n
I tend to describe the \u201cdeep web\u201d as those parts of the public internet that:<\/p>\n
The \u201cdark web,\u201d by contrast, are pages \u2013 some of which may also have \u201cdeep web\u201d elements \u2013 that are hosted by web servers using the anonymous web protocol called Tor.<\/p>\n
Originally developed by US Defence Department researchers to secure sensitive information, Tor was released into the public domain in 2004.<\/p>\n
Like many secure systems such as the WhatsApp messaging app, its original purpose was for good, but has also been used by criminals hiding behind the system\u2019s anonymity.<\/p>\n
Some people run Tor sites handling illicit activity, such as drug trafficking, weapons and human trafficking and even murder for hire.<\/p>\n
The US government has been interested in trying to find ways to use modern information technology and computer science to combat these criminal activities.<\/p>\n
In 2014, the Defence Advanced Research Projects Agency (more commonly known as DARPA), a part of the Defence Department, launched a program called Memex to fight human trafficking with these tools.<\/p>\n
Specifically, Memex wanted to create a search index that would help law enforcement identify human trafficking operations online \u2013 in particular by mining the deep and dark web.<\/p>\n
One of the key systems used by the project\u2019s teams of scholars, government workers and industry experts was one I helped develop, called Apache Tika.<\/p>\n
Tika is often referred to as the \u201cdigital Babel fish,\u201d a play on a creature called the \u201cBabel fish\u201d in the \u201cHitchhiker\u2019s Guide to the Galaxy\u201d book series.<\/p>\n
Once inserted into a person\u2019s ear, the Babel fish allowed her to understand any language spoken.<\/p>\n
Tika lets users understand any file and the information contained within it.<\/p>\n
When Tika examines a file, it automatically identifies what kind of file it is \u2013 such as a photo, video or audio.<\/p>\n
It does this with a curated taxonomy of information about files: their name, their extension, a sort of \u201cdigital fingerprint”.<\/p>\n
When it encounters a file whose name ends in “.MP4” for example, Tika assumes it\u2019s a video file stored in the MPEG-4 format.<\/p>\n
By directly analysing the data in the file, Tika can confirm or refute that assumption \u2013 all video, audio, image and other files must begin with specific codes saying what format their data is stored in.<\/p>\n
Once a file\u2019s type is identified, Tika uses specific tools to extract its content such as Apache PDFBox for PDF files, or Tesseract for capturing text from images.<\/p>\n
In addition to content, other forensic information or “metadata\u201d is captured including the file\u2019s creation date, who edited it last, and what language the file is authored in.<\/p>\n
From there, Tika uses advanced techniques like Named Entity Recognition (NER) to further analyse the text.<\/p>\n
NER identifies proper nouns and sentence structure, and then fits this information to databases of people, places and things, identifying not just whom the text is talking about, but where, and why they are doing it.<\/p>\n
This technique helped Tika to automatically identify offshore shell corporations (the things); where they were located; and who (people) was storing their money in them as part of the Panama Papers scandal that exposed financial corruption among global political, societal and technical leaders.<\/p>\n