• We've moved. Our community is now at ownd.sh — latest discussions and updates live there.

Looking for Large Real-World Datasets (20–40GB+) for Learning

Martyr

Well-known member
Martyr Rep
0
0
Rep
56
Martyr Vouches
0
0
0
Vouches
0
Posts
202
Likes
21
Bits
1 YEAR
1 YEAR OF SERVICE
=150Looking for Large Real-World Datasets (20–40GB+) for Learning

Guys, I’m looking to build a data pool or maybe a small data lake – not fully decided yet, it depends on the types of logs I end up working with.[conversation_history:1]

I’m specifically hunting for large, real datasets in the 20–40 GB range (or more): things like log files, telemetry, events, etc. Not garbage sample data with repeated or meaningless lines – I want something realistic enough that I can actually learn how to:

- Store and organize big datasets

- Run custom queries in (near) real time

- Get search results back in minutes at scale

[conversation_history:1]

This is purely for learning and experimentation. It will not be sold, shared as a “product,” or used commercially in any way – it’s just for personal R&D and skill-building.[conversation_history:1]

If you know:
- Public/open datasets with large log-style data

- Any sources where big realistic DB dumps are available

- Or tips on what stack to target (e.g., data lake vs “just” a big DB) for this size of data

Please drop any leads or ideas. Any pointers would really help.[conversation_history:1]
 

179,172

126,023

1,405,739

Top