AWS VPC flow logs

AWS defines flow log as:

VPC Flow Logs is a feature that enables you to capture information about the IP traffic going to and from network interfaces in your VPC. Flow log data can be published to Amazon CloudWatch Logs or Amazon S3. After you've created a flow log, you can retrieve and view its data in the chosen destination.

In simple terms, flow logs provide information about the traffic moving in and out of an instance, more specifically ENI. Flow logs can be published to S3 or CloudWatch for analysis. You can create flow logs for a VPC, a subnet, or a network interface. If you create a flow log for a VPC, each network interface in that VPC is monitored.

Some use cases

Traffic analysis
Detect anomalies
Network troubleshooting
Diagnosing connectivity issues

Default flow log format

<version> <account-id> <interface-id> <srcaddr> <dstaddr> <srcport> <dstport> <protocol> <packets> <bytes> <start> <end> <action> <log-status>

version - flow log version. Default format uses version 2
account-id - AWS account ID
interface-id - Network interface ID
srcaddr - Source IP address
dstaddr - Destination IP address
srcport - Source port of the traffic
dstport - Destination port of the traffic
protocol - Protocol number of the traffic.
packets - The number of packets transferred during the flow.
bytes - The number of bytes transferred during the flow.
start - Time in Unix seconds
end - Time in Unix seconds
action - Traffic is either accepted or rejected (ACCEPT/REJECT)
log data:
OK: Data is logging normally to S3 or CloudWatch.
NODATA: There was no network traffic to or from the network interface during the aggregation interval.
SKIPDATA: Some flow log records were skipped during the aggregation interval.

Example - Setting up flow logs for the VPC

Create a log group on CloudWatch. A log group is a group of log streams that share the same retention, monitoring, and access control settings.

CloudWatch > Log groups > Actions > Create log group

2. Create IAM role for publishing flow logs to CloudWatch. The IAM role associated with the flow log should have enough permissions to publish flow logs to CloudWatch Logs.

3. Create flow log for the VPC. Select "All" if you want to capture both Accepted and Rejected traffic. The aggregation interval is the period of time during which a particular flow is captured and aggregated into a flow log record. Assign the log group and IAM role.