A company is implementing a new containerized application on Amazon Elastic Kubernetes Service (EKS). The security team requires a solution to centralize and analyze logs from all pods across multiple EKS clusters for security auditing and incident response. The solution must support filtering, searching, and long-term retention of logs. Which combination of AWS services would best meet these requirements?
- AAmazon CloudWatch Logs and AWS Glue
- BAmazon Kinesis Data Firehose and Amazon S3
- CFluent Bit, Amazon Kinesis Data Firehose, and Amazon S3
- DAWS CloudTrail and Amazon Athena
Show answer & explanationAnswer & explanation
Correct answer: C. Fluent Bit, Amazon Kinesis Data Firehose, and Amazon S3
Fluent Bit is a lightweight log processor and forwarder that can run as a DaemonSet in EKS to collect container logs. Kinesis Data Firehose can then ingest these logs, transform them, and deliver them to Amazon S3 for long-term retention. S3 provides scalable and durable storage, while the combination with other services (like Athena or CloudWatch Logs Insights) can be used for filtering and searching. This approach provides a robust, scalable, and cost-effective centralized logging solution for EKS.
Why the other options are wrong
- A. CloudWatch Logs can collect logs, but for long-term retention and advanced analysis of large volumes of container logs, direct integration with S3 via Firehose is often more cost-effective and scalable. AWS Glue is for ETL, not direct log ingestion and storage.
- B. While Kinesis Data Firehose and S3 are crucial for ingestion and storage, there needs to be a mechanism to collect logs from EKS pods and send them to Firehose. This option misses the critical log collector component.
- D. AWS CloudTrail records AWS API calls and related events, not application logs from EKS pods. Amazon Athena can query data in S3 but doesn't handle log collection or ingestion directly.
EKS Container Logging
Collecting, centralizing, and analyzing logs from applications running in Amazon EKS containers for monitoring, troubleshooting, and security auditing.
- Requires a log collector agent within the EKS cluster.
- Often uses Kinesis Data Firehose for ingestion.
- Amazon S3 is a common destination for long-term storage.
Memory trick: Fluent Bit ships logs to Firehose, which then stores them safely in S3.