We are at the end!

We are at the end!

Blog 8 of the 8 Blog series

Β·

6 min read

After all this effort we are done with our tiny todo application which follows "realtime sync". In this last blog let's discuss a few pointers that you shall keep in mind while building these kind of applications and let's also write some code around error handling.

Things to note

  1. In such applications where every single possible mutation/interaction by the user on the app is optimistic, it is important to note that all checks should be done on frontend

    • It is feasible to do so, because you have at your exposure all the data you need to perform validation inside replicache itself

    • for example in a traditional application when you submit a form to create a task, it shows a loading indicator and sends your request to the server where all validation happens and task get's created

      • You either get a success or an error flag on client after the mutation has happened on the server.

      • But in our kind of applications we will optimistically render changes and then if our server fails, reverting is very bad UX

    • example of frontend validation

    • Like in the above screen shot we do all kind of checks before actually updating the todo on the client side mutator itself (you can find it inside todo.mutator.ts inside web app). Same set of checks are done on the backend as well for sanity and guarding agains malicious requests

    • If some mutation can't be done on replicache (like authentication or some action that needs to check for a global unique constraint like an organisation slug etc) should be performed by traditional api routes

  2. Next thing to keep in mind is your push and pull endpoints both are transactional with a very high isolation level, so you should write backend services keeping one thumb rule in mind, "Go the extra mile for performance"

    • Rule of thumb while transactional operations are

      1. Don't call any external apis or services inside a transaction because they can't be rolled back

        • In-DB queues are your friends if you want to conduct background jobs inside your mutations. Implementation of this is beyond the scope of this series. But maybe I will add another blog in future demonstrating how to setup In-DB queue and consumers to do background jobs.
      2. Make your transactional services as efficient as possible, (writing more efficient SQL queries).

      3. And on production database server, api server and your redis instance should be in same private network (VPC) --> "more performant and secure"

Finally let's write down some error handling for some unforeseen errors. In our current setup

Say a error happens while creating a todo, on the immediate pull it reverts back automatically

In this case I have updated the create function inside todo.service.ts file of our api to always throw an error

 async create({ args: { id, text }, userId }: MutatorContext & { args: TodoCreateType }) {
    throw new AppError({
      code: "UNAUTHORIZED",
      message: "You are not authorized to perform this operation",
    });
  }

Now in our push handler, because of the way we have structured it,

  1. First server tries to perform the mutation and if succedds it's done

  2. If fails due to some error, we again process the mutation with error mode true which just increments the last mutation id of client so replicache client does not keep on retrying to run this mutation

  3. If it's a deadlock error we try in the same transaction around 10 times (it mostly passes in these many attempts if there is a deadlock) if didn't pass again goes to catch and last mutationID is updated and we return a response

  4. Worst case scenario if server is down or database is down only then this route throws an error and a non 200 response

  5. So as you can see we need to catch and send the error along with the 200 response

push: RequestHandler = async (
    req: Request<object, object, PushRequestType["body"]>,
    res: Response,
    next: NextFunction,
  ) => {
    const userId = req.user.id;
    // init an errors array to return error messages to clients
    const errors: {
      mutationName: string;
      errorMessage: string;
      errorCode: string;
    }[] = [];
    try {
      const push = req.body;
      for (const mutation of push.mutations) {
        try {
          await ReplicacheService.processMutation({
            clientGroupID: push.clientGroupID,
            errorMode: false,
            mutation,
            userId,
          });
        } catch (error) {
          await ReplicacheService.processMutation({
            clientGroupID: push.clientGroupID,
            errorMode: true,
            mutation,
            userId,
          });
          // if error is an AppError means we are throwing this error because some check failed
          if (error instanceof AppError) {
            errors.push({
              mutationName: mutation.name,
              errorMessage: error.message,
              errorCode: error.code,
            });
          }
        }
      }
      return res.status(200).json({
        success: errors.length === 0,
        errors,
      });
    } catch (error) {
      if (error instanceof AppError) {
        return next(error);
      }
      logger.error(error);
      return next(
        new AppError({
          code: "INTERNAL_SERVER_ERROR",
          message:
            "Failed to push data to the server, due to an internal error. Please try again later.",
        }),
      );
    } finally {
      await sendPoke({ userId });
    }
  };

Now we use this errors array to actually render error messages in from of a toast in case of any unforeseen errors. "Check the pusher function where we call toast".

import { useAbly } from "ably/react";
import { AxiosError } from "axios";
import { nanoid } from "nanoid";
import * as React from "react";
import toast from "react-hot-toast";
import { create } from "zustand";
import { immer } from "zustand/middleware/immer";

import { type M, type MutatorType, type PullResponseOKV1, Replicache } from "@repo/models";

import { api } from "~/lib/api";

import { env } from "~/env";
import { useUser } from "~/hook/user-user";
import { clientMutators } from "~/mutators";

type State = {
  // M is a type for all mutators in the app (client and server)
  rep: Replicache<M<MutatorType.CLIENT>> | null;
};

type Actions = {
  setRep: (rep: Replicache<M<MutatorType.CLIENT>>) => void;
};

const useReplicacheStore = create<State & Actions>()(
  immer((set) => ({
    rep: null,
    setRep: (rep) => set({ rep }),
  })),
);

export const useReplicache = () => {
  return { rep: useReplicacheStore((state) => state.rep) };
};

export const useLoadReplicache = () => {
  const { data } = useUser();
  const user = data?.user;
  const { rep, setRep } = useReplicacheStore((state) => state);
  const ably = useAbly();

  React.useEffect(() => {
    if (!user?.id) return;
    const iid = nanoid();

    const r = new Replicache({
      name: user.id,
      licenseKey: env.NEXT_PUBLIC_REPLICACHE_LICENSE_KEY,
      mutators: clientMutators(user.id),
      schemaVersion: env.NEXT_PUBLIC_SCHEMA_VERSION ?? "1",
    });

    r.pusher = async (opts) => {
      try {
        const response = await api.replicachePush(opts, iid);

        // if not a success response throw error in form of a toast    
        if (!response.data.success) {
          response.data.errors.forEach((error) => {
            console.error(`Error processing mutation ${error.mutationName}: ${error.errorMessage}`);
            toast.error(`Error processing mutation ${error.mutationName}: ${error.errorMessage}`);
          });
        }

        return {
          httpRequestInfo: {
            httpStatusCode: response.status,
            errorMessage: "",
          },
        };
      } catch (error) {
        if (error instanceof AxiosError)
          return {
            httpRequestInfo: {
              httpStatusCode: error.status ?? 500,
              errorMessage: error.message,
            },
          };
        return {
          httpRequestInfo: {
            httpStatusCode: 500,
            errorMessage: "Unknown error",
          },
        };
      }
    };

    r.puller = async (opts) => {
      try {
        const response = await api.replicachePull(opts, iid);
        return {
          response: response.data as PullResponseOKV1,
          httpRequestInfo: {
            errorMessage: "",
            httpStatusCode: response.status,
          },
        };
      } catch (error) {
        if (error instanceof AxiosError)
          return {
            httpRequestInfo: {
              httpStatusCode: error.status ?? 500,
              errorMessage: error.message,
            },
          };
        return {
          httpRequestInfo: {
            httpStatusCode: 500,
            errorMessage: "Unknown error",
          },
        };
      }
    };

    setRep(r);

    return () => {
      void r.close();
    };
  }, [setRep, user?.id]);

  React.useEffect(() => {
    if (!rep || !user?.id) return;
    const channel = ably.channels.get(`replicache:${user.id}`);
    channel.subscribe(() => {
      void rep?.pull();
    });

    return () => {
      const channel = ably.channels.get(`replicache:${user.id}`);
      channel.unsubscribe();
    };
  }, [rep, ably.channels, user?.id]);
};

With this we finally reach the end of this blog series. Hope you enjoyed the series and probably learnt a thing or two about building realtime sync apps.

Meet you guys in the next one πŸ‘‹πŸΌπŸ‘‹πŸΌ

Β