An introduction to VPC flow logs

Flow logging is a free AWS VPC feature that can capture and log traffic transiting through your AWS estate’s network interfaces. Flow logs capture all traffic by default, but can be configured to only capture accepted or rejected traffic. Similar traffic (e.g., multiple packets from the same source to the same destination) across an interface is aggregated over an interval and combined into a flow log record, as seen below. You’ll learn how to interpret these records later in the lab.

Each log entry or record contains information such as the account ID and interface ID of the NI in question, the source of traffic (the IP address and port), and the destination. Records also show how many packets and bytes were sent during the aggregation interval and the IANA protocol number of the traffic. You can view more information on all available fields here – version two fields are included in each record by default, and fields from version three or higher must be added with a custom log record format.

Flow log records can be stored in CloudWatch log groups or S3 buckets and retained for auditing and investigation purposes. Integration with CloudWatch logs means that the CloudWatch Alarms feature can be used with flow logs, enabling monitoring and alerting on network traffic through your network interfaces.

Creating flow logs

There's no console section dedicated to flow logs, so they must be created and viewed within the resources they're attached to. In the VPC console, you can select a VPC or a subnet (or multiple) and select Create flow log from the Actions dropdown. An example of this can be seen in the screenshot below. Similarly, in the Network interfaces section of the EC2 console, you can select one or more interfaces to create a flow log on, again using the Actions dropdown.

It’s worth noting that if you've opted to create a flow log for multiple resources, AWS creates one flow log for each selected resource, with matching configurations as prescribed by the user during creation.

Create a CloudWatch log group called eu-west-1b-logs. Leave all settings as default.

Now, Create a flow log called eu-west-1b-all-traffic for the default subnet in the eu-west-1b availability zone. It should capture all traffic, on a maximum aggregation interval of one minute, and send traffic to your eu-west-1b-logs CloudWatch log group. Use the existing metrolio-flow-logs-4df7f3c3 IAM role and leave everything else as default.

Next, Create a flow log called web-traffic, capturing only accepted traffic for both of the existing EC2 network interfaces in eu-west-1. The aggregation interval should be ten minutes and logs should be pushed to the web folder of your S3 bucket. Leave everything else as default. Note: The bucket ARN is arn:aws:s3:::metrolio-eu-ff578811.