Guides

Archive casts

Archiving Farcaster data with Neynar

Suggest Edits

Casts in the Farcaster protocol are pruned when user runs out of storage. This guide demonstrates how to archive casts of a specific FID with the Neynar SDK.

Check out this Getting started guide to learn how to set up your environment and get an API key.

Check out this example repository to see the code in action.

First, initialize the client:

// npm i @neynar/nodejs-sdk
import { NeynarAPIClient } from "@neynar/nodejs-sdk";

// make sure to set your NEYNAR_API_KEY .env
// don't have an API key yet? get one at neynar.com
const client = new NeynarAPIClient(process.env.NEYNAR_API_KEY);

Next, let's make a function to clean the incoming casts:

const parser = (cast) => {
  return {
    fid: parseInt(cast.author.fid),
    parentFid: parseInt(cast.parentAuthor.fid)
      ? parseInt(cast.parentAuthor.fid)
      : undefined,
    hash: cast.hash || undefined,
    threadHash: cast.threadHash || undefined,
    parentHash: cast.parentHash || undefined,
    parentUrl: cast.parentUrl || undefined,
    text: cast.text || undefined,
  };
};

Then, the function to archive the casts:

const dumpCast = (cast) => {
  const parsed = parser(cast);
  const data = `${JSON.stringify(parsed)}\n`;
  fs.appendFileSync("data.ndjson", data);
};

Finally, let's fetch the casts and archive them:

const fetchAndDump = async (fid, cursor) => {
  const data = await client.fetchAllCastsCreatedByUser(fid, {
    limit: 150,
    cursor,
  });
  data.result.casts.map(dumpCast);

  // If there is no next cursor, we are done
  if (data.result.next.cursor === null) return;
  await fetchAndDump(fid, data.result.next.cursor);
};

// archive all @rish.eth's casts in a file called data.ndjson
const fid = 194;
fetchAndDump(fid);

Result: a file called data.ndjson with all the casts of the user with FID 194.

It looks something like this:

{"fid":194,"parentFid":3,"hash":"0x544421c091f5af9d1610de0ae223b52602dd631e","threadHash":"0xb0758588c9412f72efe7e703e9d0cb5f2d0a6cfd","parentHash":"0xb0758588c9412f72efe7e703e9d0cb5f2d0a6cfd","text":"that order is pretty key"}
{"fid":194,"parentFid":194,"hash":"0x98f52d36161f3d0c8dee6e242936c431face35f0","threadHash":"0x5727a985687c10b6a37e9439b2b7a3ce141c6237","parentHash":"0xcb6cab80cc7d7a2ca957d1c95c9a3459f9e3a9dc","text":"turns out not an email spam issue 😮‍💨, email typo :)"}
{"fid":194,"parentFid":20071,"hash":"0xcb6cab80cc7d7a2ca957d1c95c9a3459f9e3a9dc","threadHash":"0x5727a985687c10b6a37e9439b2b7a3ce141c6237","parentHash":"0xf34c18b87f8eaca2cb72131a0c0429a48b66ef52","text":"hmm interesting. our system shows the email as sent. Maybe we're getting marked as spam now? 🤦🏽‍♂️\n\nLet me DM you on telegram"}
{"fid":194,"parentFid":20071,"hash":"0x62c484064c9ca1177f8addb56bdaffdbede97a29","threadHash":"0x5727a985687c10b6a37e9439b2b7a3ce141c6237","parentHash":"0x7af582a591575acc474fa1f8c52a2a03258986b9","text":"are you still waiting on this? you should have gotten the email within the first minute. we automated this last week so there's no wait anymore. lmk if you're still having issues :)"}
{"fid":194,"parentFid":3,"hash":"0xbc63b955c40ace8aca4b1608115fd12f643395b1","threadHash":"0x5727a985687c10b6a37e9439b2b7a3ce141c6237","parentHash":"0x5727a985687c10b6a37e9439b2b7a3ce141c6237","text":"@bountybot adding 150 USDC to this bounty \n\nfor anyone building on this, please reach out with any questions. We've always wanted to do this but haven't been able to prioritize. Think this can be quite impactful! :)"}

That's it! You now can save that in S3 or IPFS for long-term archival!

🚀

Ready to start building?

Get your subscription at neynar.com and reach out to us on Telegram with any questions!

Updated 2 months ago