Guides

From Parquet exports

Ingest farcaster data into your database or data warehouse from parquet exports

Suggest Edits

📘

Contact for setup and pricing

Frequency

  • Full exports are created every day
  • Incremental exports are created every 30 mins

Ingestion

See How to ingest

Full archives: s3://tf-premium-parquet/public-postgres/farcaster/v2/full/

Incremental archives: s3://tf-premium-parquet/public-postgres/farcaster/v2/incremental/

Notifications using SNS

You can subscribe to SNS to get notifications whenever a new file is added to the s3 bucket: arn:aws:sns:us-east-1:252622759102:tf-s3-premium-parquet

Schema

Schema

Notes

  1. Data is exported every 30 mins and once a day. You can start by downloading a daily snapshot and then ingesting the incremental 30m files.
  2. Originally timestamps were stored with nanoseconds, but as of April 23, 2024 they are all using microseconds.
  3. The “messages” table is very very large and we are not currently exporting it. Please talk with us if you need the table and we can figure out the best solution for you.

Updated 3 months ago