AWS VPC flow logs

Suresh Vina -

AWS defines flow log as:

VPC Flow Logs is a feature that enables you to capture information about the IP traffic going to and from network interfaces in your VPC. Flow log data can be published to Amazon CloudWatch Logs or Amazon S3. After you've created a flow log, you can retrieve and view its data in the chosen destination.

In simple terms, flow logs provide information about the traffic moving in and out of an instance, more specifically ENI. Flow logs can be published to S3 or CloudWatch for analysis. You can create flow logs for a VPC, a subnet, or a network interface. If you create a flow log for a VPC, each network interface in that  VPC is monitored.

Some use cases

  • Traffic analysis
  • Detect anomalies
  • Network troubleshooting
  • Diagnosing connectivity issues

Default flow log format

<version> <account-id> <interface-id> <srcaddr> <dstaddr> <srcport> <dstport> <protocol> <packets> <bytes> <start> <end> <action> <log-status>
  • version - flow log version. Default format uses version 2
  • account-id - AWS account ID
  • interface-id - Network interface ID
  • srcaddr - Source IP address
  • dstaddr - Destination IP address
  • srcport - Source port of the traffic
  • dstport - Destination port of the traffic
  • protocol - Protocol number of the traffic.
  • packets - The number of packets transferred during the flow.
  • bytes - The number of bytes transferred during the flow.
  • start - Time in Unix seconds
  • end - Time in Unix seconds
  • action - Traffic is either accepted or rejected (ACCEPT/REJECT)
  • log data:
    OK: Data is logging normally to S3 or CloudWatch.
    NODATA: There was no network traffic to or from the network interface during the aggregation interval.
    SKIPDATA: Some flow log records were skipped during the aggregation interval.

Example - Setting up flow logs for the VPC

  1. Create a log group on CloudWatch. A log group is a group of log streams that share the same retention, monitoring, and access control settings.

CloudWatch > Log groups > Actions > Create log group

2. Create IAM role for publishing flow logs to CloudWatch. The IAM role associated with the flow log should have enough permissions to publish flow logs to CloudWatch Logs.

3. Create flow log for the VPC. Select "All" if you want to capture both Accepted and Rejected traffic. The aggregation interval is the period of time during which a particular flow is captured and aggregated into a flow log record. Assign the log group and IAM role.

Analysis

  • I have an instance running with the IP of 10.10.1.10 and listening on port 443.
  • Security group allows ANY IP to access the instance on port 443 (web server)
Someone tried to access the instance from 167.71.109.229 on port 8088 which was rejected by the Security group.
  • Source IP - 167.71.109.229
  • Destination IP - 10.10.1.10
  • Source port - 59754
  • Destination port - 8088
  • Protocol number - 6 (TCP)
  • Number of packets - 1
  • Packet size - 40 bytes
  • Start time - 1587749363 is equivalent to 04/24/2020 6:29pm (GMT)
  • End time - 1587749422 is equivalent to 04/24/2020 6:30pm (GMT)
  • Action - Rejected
  • Log data - Published to CloudWatch

Traffic going to port 443 was allowed through.

Create flow log only for a specific Network Interface

Go to EC2 > Network Interfaces >> Select the ENI >> Actions > Create Flow log

Reference

VPC Flow Logs - Amazon Virtual Private Cloud
Create a flow log to capture information about IP traffic going to and from your network interfaces.
Amazon Virtual Private Cloud

Buy me a coffee