Data and Analytics on AWS: Athena, Glue, Kinesis, EMR, Redshift, QuickSight and OpenSearch
The Typical Analytics Flow
text
Sources (apps, logs, IoT) ─► Ingest (Kinesis / Firehose / DMS)
─► Store (S3 data lake) ─► Prepare (Glue / EMR)
─► Analyze (Athena / Redshift / OpenSearch) ─► Visualize (QuickSight)
| Service | What it is | Think of it as |
|---|---|---|
| S3 | Cheap, durable storage for any data | The data lake |
| AWS Glue | Serverless ETL, crawlers and Data Catalog | Data cleaner and index |
| Amazon Athena | Run SQL directly on files in S3. Pay per data scanned | SQL on your files |
| Kinesis Data Streams | Real-time stream you read with your own consumers | A live conveyor belt |
| Kinesis Data Firehose | Load streaming data into S3, Redshift, OpenSearch with no code | Delivery truck |
| Amazon EMR | Managed Hadoop and Spark clusters (or serverless) | Big data processing |
| Amazon Redshift | Columnar data warehouse for fast complex SQL | Warehouse for BI |
| Amazon QuickSight | BI dashboards and reports | Charts for managers |
| OpenSearch Service | Search and log analytics | Search engine |
| Lake Formation | Central permissions and setup for the data lake | Data lake security guard |
| MSK | Managed Apache Kafka | Kafka without servers to manage |
Athena in Five Minutes
sql
CREATE EXTERNAL TABLE sales (
order_id string, amount double, country string, order_date date
) ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
LOCATION 's3://my-data-lake/sales/' TBLPROPERTIES ('skip.header.line.count'='1');
SELECT country, SUM(amount) AS revenue
FROM sales WHERE order_date >= DATE '2026-11-01'
GROUP BY country ORDER BY revenue DESC;
- Store data in Parquet and partition by date to scan less and pay less
- Save query results to an S3 location
- No servers to manage
Glue Crawler and ETL
bash
aws glue create-database --database-input Name=lake
aws glue create-crawler --name sales-crawler --role AWSGlueServiceRole-demo --database-name lake --targets '{"S3Targets":[{"Path":"s3://my-data-lake/sales/"}]}'
aws glue start-crawler --name sales-crawler
aws glue get-tables --database-name lake --query 'TableList[].Name'
Streaming With Kinesis
bash
aws kinesis create-stream --stream-name clicks --shard-count 1
aws kinesis put-record --stream-name clicks --partition-key user1 --data "$(echo '{"page":"/home"}' | base64)"
aws kinesis get-shard-iterator --stream-name clicks --shard-id shardId-000000000000 --shard-iterator-type TRIM_HORIZON
Which One When?
| Need | Choose |
|---|---|
| Quick SQL on S3 files, no setup | Athena |
| Fast dashboards on large structured data every day | Redshift |
| Custom Spark or Hadoop jobs | EMR (or Glue for simple ETL) |
| Real-time ingestion and processing | Kinesis Data Streams / MSK |
| Land streaming data in S3 with no code | Firehose |
| Search and log exploration | OpenSearch |
| Business dashboards | QuickSight |
Practice
- Upload a CSV to S3 and query it with Athena
- Convert it to Parquet and compare the data scanned
- Create a Glue crawler and look at the table it creates
- Make a QuickSight chart from the Athena table
- Delete streams and clusters when done
Conclusion
S3 is the data lake. Ingest with Kinesis or Firehose, prepare with Glue or EMR, analyze with Athena, Redshift or OpenSearch and visualize with QuickSight. Use Parquet and partitions to save money.
Quick Summary: flow; services; Athena; Glue; Kinesis; choosing.

