Elastic / ELK Integration

The Infracast Elastic integration exports findings and discovery events directly to an Elasticsearch cluster. Findings can then be visualized in Kibana, correlated with other log data, and used to build custom security dashboards.

Prerequisites

• Elasticsearch 7.x or 8.x (self-hosted or Elastic Cloud)
• An Elasticsearch API key or username/password with write access to the target index
• Network connectivity from Infracast to the Elasticsearch cluster (TCP/9200 for self-hosted, TCP/443 for Elastic Cloud)

Creating an Elasticsearch API Key

Elastic Cloud or self-hosted Elasticsearch (8.x):

# Create an index pattern first (optional but recommended)
curl -X PUT "https://elasticsearch.example.com:9200/infracast-findings" \
  -H "Content-Type: application/json" \
  -u elastic:password \
  -d '{
    "settings": {
      "number_of_shards": 1,
      "number_of_replicas": 1
    }
  }'

# Create an API key scoped to the infracast-findings index
curl -X POST "https://elasticsearch.example.com:9200/_security/api_key" \
  -H "Content-Type: application/json" \
  -u elastic:password \
  -d '{
    "name": "infracast-discovery",
    "role_descriptors": {
      "infracast_writer": {
        "cluster": ["monitor"],
        "indices": [
          {
            "names": ["infracast-*"],
            "privileges": ["create_index", "write", "read"]
          }
        ]
      }
    }
  }'

The response includes an encoded field — this is the API key value to use in Infracast.

tip

Scope the API key to only the infracast-* index pattern. Infracast does not need cluster-admin or read access to other indexes.

Configuring the Integration

Navigate to Settings → Integrations → Add Integration → Elastic:

Field	Description
Cluster URL	Elasticsearch endpoint, e.g., https://elasticsearch.example.com:9200 or Elastic Cloud URL
API Key	Base64-encoded API key (the encoded value from the API key creation response)
Index	Target index name (default: infracast-findings)
TLS Verify	Verify the Elasticsearch TLS certificate
Events to Export	Select which events to send: findings, discovery completions, asset changes
Minimum Severity	Only export findings at or above this severity (e.g., Medium)

Document Format

Each finding is indexed as an Elasticsearch document:

{
  "@timestamp": "2024-06-10T12:00:00Z",
  "event": {
    "type": "finding",
    "severity": "critical",
    "category": "vulnerability"
  },
  "infracast": {
    "finding_id": "fnd-abc123",
    "title": "Critical vulnerability CVE-2024-1234",
    "severity": "critical",
    "asset_id": "aws.ec2.instance.i-0abc123",
    "asset_name": "web-server-01",
    "asset_type": "aws.ec2.instance",
    "plugin": "aws",
    "cve": "CVE-2024-1234",
    "cvss_score": 9.8,
    "remediation": "Apply patch X.",
    "tenant_id": "tenant-xyz"
  },
  "host": {
    "name": "web-server-01"
  }
}

The document structure follows Elastic Common Schema (ECS) conventions for compatibility with standard Kibana dashboards and detection rules.

Kibana Dashboard Setup

After connecting Infracast to Elasticsearch:

1. Open Kibana → Stack Management → Index Patterns
2. Create an index pattern matching infracast-*
3. Set @timestamp as the time field
4. Navigate to Discover to explore findings data
5. Use Dashboard → Create new to build custom visualizations using the infracast.* fields

Troubleshooting

401 Unauthorized

Symptom: Error: Elasticsearch request failed: 401 Unauthorized

Checks:

1. Verify the API key is correct and has not been invalidated:

   curl -H "Authorization: ApiKey YOUR_ENCODED_KEY" \
     https://elasticsearch.example.com:9200/_security/_authenticate

2. If using username/password instead of API key, verify the credentials

403 Forbidden on index write

Symptom: Error: index write failed: 403 Forbidden

Checks:

1. Verify the API key has write privilege on the infracast-* index pattern
2. If using ILM (Index Lifecycle Management), the write alias must point to the active index

TLS certificate errors (Elastic Cloud)

Elastic Cloud uses valid public TLS certificates. Set TLS Verify to true. If using a self-hosted cluster with a custom CA:

# Get the CA cert from your Elasticsearch deployment
curl -sk https://elasticsearch.example.com:9200 -v 2>&1 | grep "issuer"

Provide the CA certificate file in the Infracast credential configuration.

Events indexed but Kibana shows no data

Symptom: Index exists with documents but Kibana Discover shows no results

Checks:

1. Verify the Kibana index pattern is created: Stack Management → Index Patterns
2. Ensure the @timestamp field is set as the time field for the index pattern
3. Check the time range selector in Kibana — it defaults to the last 15 minutes