Output

url

URL of the post's page.

No

Yes

String

uuid

A unique identifier for the cyber document (e.g., post, comment, message).

No

Yes

String

parent_uuid

The unique identifier of the parent document.

• For structured data: represents the thread URL.

• For unstructured data: represents the referring URL. (partially)

No

Yes

String

title

The title of the post.

Yes

Yes.

String

text

The content of the post.

•For standard posts: a plain text string.

• For chat data: a JSON-formatted string containing an array of message objects (see structure below).

No

Yes

String

author

The author (username) of the post.

• For standard posts: a string representing the username.

• For chat data: an array of strings, each representing a participant’s username.

Yes

Yes

String / Array[String]

language

The primary language detected in the post text.

No

Yes

String

referring_url

The URL that referred the crawler to this post.

Yes

Yes

String

referring_file_type

The file type of the referring URL.

Yes

Yes

String

rating

Obsolete. Left for backward Compatibility.

(rating is a floating number between 0.0 to 5.0.)

Yes

Yes

Float

published

The publication date and time of the post.

• For structured data: This field is mandatory.
• For unstructured data: This field is not mandatory.

Format: yyyy-MM-dd'T'HH:mm:ss.SSSXXX

Yes

Yes

Date

ord_in_thread

The post’s position in the thread.
0 indicates the first post.

This field is mandatory for structured data and for standard posts (non-chats).

Yes

Yes

Integer

crawled

The date and time when the post was crawled.

Format: yyyy-MM-dd'T'HH:mm:ss.SSSXXX

No

Yes

Date

updated

The date and time when the post was updated.

Format: yyyy-MM-dd'T'HH:mm:ss.SSSXXX

Yes

No

Date

cache_link

A link to a cached HTML snapshot of the original post page.

Yes

No

String

date

The date and time the message was published.

Format: yyyy-MM-dd'T'HH:mm:ss.SSSXXX

No

Yes (within text field)

Date

text

The text within the message.

Yes

Yes (within text field)

String

user

The username of the message author.

Yes

Yes (within text field)

String

message_id

The unique identifier of the message.

Available only on Telegram.

Yes

Yes (within text field)

String

forwarded

A Boolean value (true/false) indicating if the message was forwarded from a different group/channel.

Available only on Telegram.

Yes

Yes (within text field)

Boolean

forwarded_from

The group/channel from which the message was forwarded.

Available only on Telegram.

Yes

Yes (within text field)

String [URL]

file_info.file_name

The name of a file attached to the message.

Available only on Telegram.

Yes

Yes (within text field)

String

file_info.file_size

The size of a file attached to the message.

Available only on Telegram.

Yes

Yes (within text field)

String

file_info.file_type

The type of file attached to the message.

Available only on Telegram.

Yes

Yes (within text field)

String

message_link

The unique link assigned to the message.

Available only on Telegram.

Yes

Yes (within text field)

String [URL]

name

The name of the site where the post was published.

Yes

Yes

String

is_live

Obsolete. Retained for backward compatibility. Value is always true.

No

No

Boolean

domain

The domain of the site.

No

Yes.

String

current_domain

The domain used at the time the post was scanned.

This field is mandatory for structured data.

Yes

Yes.

String

type

The general type of the site.

Possible values include:

• news
• blogs
• discussions
• chat
• market
• paste
• datastore
• social_media
• imageboard
• ransomware_blog

This field is mandatory for structured data.

Yes

Yes

String

categories

Site-level categories based on the type of content:

For standard posts:
• cyber_threat_intelligence
• financial_crime
• illicit_trafficking
• extremism_misinformation
• brand_monitoring

For Telegram data only:
• group
• channel

For more details , please refer to this page .

Yes

Yes

Array[String]

url

The link to the thread or conversation.

• For standard posts: URL of the thread.

• For chat data: The original URL of the group or channel when added to coverage. This value is permanent.

This field is mandatory for structured data.

Yes

Yes

String

uuid

A unique identifier representing the thread.

This field is mandatory for structured data.

Yes

Yes

String

title

The title of the thread.

Yes

Yes

String

full_title

The full title of the thread.

Yes

No

String

published

The date and time the thread was originally published.

Format: yyyy-MM-dd'T'HH:mm:ss.SSSXXX

This field is mandatory for structured data.

Yes

Yes

Date

site_section

A URL to the section on the site where the thread was created.

Yes

Yes

String

section_title

The title of the section on the site where the thread was created.

Yes

Yes.

String

replies_count

The number of replies to the main post in the thread.

The default is 0 for chat data.
This field is mandatory for structured data.

Yes

Yes

Integer

participants_count

The number of unique participants who contributed to the thread.
This field is mandatory for structured data.

Yes

Yes

Integer

spam_score

Obsolete. Retained for backward compatibility.

No

No

Float

main_image

A URL to the main image associated with the thread or post.

Yes

No

URL [String]

external_links

A list of external URLs mentioned within the post.

Yes

Yes.

Array[String]

external_videos

A list of video URLs found in the post.

Yes

Yes.

Array[String]

external_images

A structured array of objects describing images found in the post.

See External Images Array of Objects below for details.

Yes

Yes.

Array of Objects

file_links

A list of file download URLs extracted from the post.

Yes

Yes.

Array[String]

file_type

The file type of the page.

Possible values:

• html
• zip
• rar
• tar
• 7z
• pdf
• txt
• xls
• xlsx
• doc
• docx
• sql

Yes

Yes

String

network

The dark/decentralized network where the post or page was found.
Possible values:

• tor
• openweb
• i2P
• telegram
• discord
• icq
• public_chat

No

Yes

String

required_login

Indicates whether a password or login is required to access the content (true or false).

No

Yes

Boolean

categories

One or more cyber categeries that the post/page content belong to.
Categories include:

• hacking
• data_breach
• carding
• crypto
• phishing
• malware
• ransomware
• stealer_logs
• counterfeit
• gaming
• extremism
• drugs
• weapons
• sexual

For more details , please refer to this page.

Yes

Yes

Array[String]

• persons

One or more individual people entities found in the post.

Currently not supported in chat data.

Yes

Yes

Object

persons.value

Person names extracted from the post

Optional

Yes

Array[String]

persons.count

Number of people mentioned in the post

Optional

Yes

Integer

• organizations

One or more organization entities found in the post.

Currently not supported in chat data.

Yes

Yes

Object

organizations.value

Organization names extracted from the post

Optional

Yes

Array[String]

organizations.count

Number of organizations mentioned in the post

Optional

Yes

Integer

• locations

One or more location entities found in the post.

Currently not supported in chat data.

Yes

Yes

Object

locations.value

Location names extracted from the post

Optional

Yes

Array[String]

locations.count

Number of locations mentioned in the post

Optional

Yes

Integer

emails

One or more email address entities found in the post

Yes

Yes

Object

emails.value

Email addresses extracted from the post

Optional

Yes

Array[String]

emails.count

Number of emails mentioned in the post

Optional

Yes

Integer

phones

One or more phone number entities found in the post

Yes

Yes

Object

phones.value

Phone numbers extracted from the post

Optional

Yes

Array[String]

phone.count

Number of phone numbers extracted from the post

Optional

Yes

Integer

credit_cards

One or more credit cards entities found in the post

Yes

Yes

Object

credit_cards.value

Credit card numbers extracted from the post

Optional

Yes

Array[String]

credit_cards.count

Number of credit card numbers extracted from the post

Optional

Yes

Integer

ssns

One or more social security number (SSN) entities found in the post

Yes

Yes

Object

ssns.value

SSN numbers extracted from the post

Optional

Yes

Array[String]

ssns.count

Number of SSNs extracted from the post

Optional

Yes

Integer

wallet_ids

One or more crypto address entities found in the post

Yes

Yes

Object

wallet_ids.ids

Crypto addresses extracted from the post

Optional

Yes

Array of Objects

wallet_ids.ids.value

Crypto address value

Optional

Yes

String

wallet_ids.ids.type

Crypto currency type

Optional

No

String

wallet_ids.count

Number of crypto addresses extracted from the post

Optional

Yes

Integer

ips

One or more IP entities found in the post

Yes

Yes

Object

ips.value

IP addresses extracted from the post

Optional

Yes

Array[String]

ips.count

Number of IP addresses extracted from the post

Optional

Yes

Integer

domains

One or more domain entities found in the post

Yes

Yes

Object

domains.value

Domain addresses extracted from the post

Optional

Yes

Array[String]

domains.count

Number of domain addresses extracted from the post

Optional

Yes

Integer

cve

One or more CVE Ids entities found in the post/page

Yes

Yes

Object

cve.value

CVEs extracted from the post

Optional

Yes

Array[String]

cve.count

Number of CVEs extracted from the post

Optional

Yes

Integer

cyber_risk

Risk score assigned to the post, ranging from 0 to 10 (where 10 indicates the highest risk level)

Yes

Object

cyber_risk.value

The overall cyber risk score assigned to the post, considering both site and content factors

Yes

Yes

Integer

cyber_risk.site_risk

Risk score attributed to the website where the post was published

Yes

Yes

Integer

cyber_risk.content_risk

Risk score assigned to the content of post itself

Yes

Yes

Integer