```yaml
---
slug: mocking-aws-with-msw
published: 2024-03-06T18:49:00-08:00
description: A Better Way to Mock AWS
changelog: ~
---
```

We use AWS pretty heavily at Taskless, with their compute-on-demand making it easy for us to scale our bi-directional API Gateway from zero to thousands of concurrent requests with no additional effort. It's truly wonderful; that is, until it's time to write tests.

The path most developers go is to grab the [aws-sdk-client-mock](https://github.com/m-radzikowski/aws-sdk-client-mock) library, which works really well assuming you don't have presigning, are using AWS Timestream, and can keep all your AWS libraries on the exact same version to avoid type conflicts.

I'm also assuming you're here because you either need pre-signing, are using AWS Timestream, think force-upgrading all AWS client libraries for a single fix is a difficult pill to swallow, or are otherwise jammed on a traditional mocking approach. The good news is, there **is** another way. We just have to move from the node / module layer down to the network layer.

## What Are You Testing, Really?

First, the realization. We don't actually care what the `@aws-sdk/*` libraries do.

There's a lot of internals in the AWS client libraries (middleware, smithy, signing, etc). Every one of them has their own abstractions, inputs, outputs, and naming conventions. Some concepts, like middleware, are impossible to mock. None of that matters though. What we **do** care about is how our app reacts to network conditions, including those made by the SDK libraries.

And **that** we can test.

* We can introduce network delays and ensure the libraries we depend on handle these events well
* We can simulate a service being down
* We use APIs instead of SDKs, which are less volatile
* We gain a better understanding about how AWS makes its network requests
* We catch if a dependency suddenly starts phoning home

So how do you go about mocking the network? There's [nock](https://www.npmjs.com/package/nock) and Mock Service Worker / [MSW](https://www.npmjs.com/package/msw). Both are excellent libraries; I just prefer MSW so I can use the same testing patterns on the node.js and browser. While these next sections are written for MSW, they can be easily applied to a nock-based setup.

## Preparing for Network Mocks

Setting up for network based mocking requires a little bit of pre-work. Specifically, we want to knock out all our AWS environment variables with dummy values; this ensures if a request ever does fall through to the network layer, it cannot trigger a billable event. When launching any node.js app, including tests, you can include a `NODE_OPTIONS` environment variable which passes additional options through to the node process. In the case of a multi-process tester like [AVA](https://github.com/avajs/ava), this ensures the options are carried to each worker. For a single-process worker like [Jest](https://jestjs.io/), any node options are passed through as if they were directly on the command line.

```json
{
    // ...
    "scripts": {
        "test": "NODE_OPTIONS='-r ./test.setup.js' ava"
    }
}
```

The above snippet adjusts our test command to include additional node options. `-r <file>` tells node.js to include a file _first_ before running any other code. In this case, we want to load a bootstrap file. Our bootstrap will take care of removing all AWS environment variables for us, first by explicitly deleting all `aws_` prefixed values, then setting suitable test values for access, secrets, and tokens. I'm using [dotenv](https://www.npmjs.com/package/dotenv) for readability, but you can also explicitly set `process.env` if you prefer.

```js
const dotenv = require("dotenv");
const util = require("node:util");

util.inspect.defaultOptions.depth = 10;

for (const key of Object.keys(process.env)) {
  if (key.toLowerCase().startsWith("aws_")) {
    delete process.env[key];
  }
}

dotenv.populate(
  process.env,
  {
    AWS_ACCESS_KEY_ID: "testing",
    AWS_SECRET_ACCESS_KEY: "testing",
    AWS_SECURITY_TOKEN: "testing",
    AWS_SESSION_TOKEN: "testing",
    AWS_DEFAULT_REGION: "us-east-1",
    AWS_REGION: "us-east-1",
    // DEBUG: "*",
  },
  { override: true }
);

```

One important reason we remove all `aws_` items is because the `AWS_PROFILE` environment variable messes with pre-signing. If the variable is set, smithy middleware will attempt to authenticate and load the profile in question. So, don't be clever; take a scorched earth approach to the `aws_*` environment variables and ensure they're all replaced.

With confidence our AWS account won't get any surprise billing, we can [follow MSW's Getting Started Guide](https://mswjs.io/docs/integrations/node) for node.js and create our handlers. We can verify MSW is working because running our tests will tell us about unhandled requests, and our first call to AWS should automatically fail.

To make it easier to mock our network requests, we'll want to create a default handler.

## The Default Handler at `*`

Unhandled requests from MSW are fine, but I find that it's not always helpful telling you where / why something fell through to the unhandled request. I recommend adding these two handlers at the end of your chain, ensuring you get actionable errors when tests try and call out to AWS.

```ts
const handlers: RequestHandler[] = [
  // force inercept of any attempts to call the local AWS Credential Provider
  http.all("http://169.254.169.254/*", () => {
    throw new Error("Attempted to call local AWS Credential Provider");
  }),

  // Provide additional info beyond MSW's default unhandled request error
  http.all("*", async (info) => {
    const request = info.request.clone();

    const debug = {
      url: request.url,
      method: request.method,
      headers: Object.fromEntries(request.headers.entries()),
      body: await request.text(),
    };

    console.error("Unhandled Request");
    console.error(JSON.stringify(debug, null, 2));
  }),
];
```

Our first handler (the `169.254...`) takes care of the AWS Credential Provider. In some scenarios, like when `AWS_PROFILE` is set to a dummy value, the credential provider is automatically called by AWS. Adding a catch-all for the credential provider will tell us immediately if AWS is attempting to verify our test credentials.

The second handler is a better debugger. Instead of a thrown error that tells you the network request came from within AWS, you can unpack the URL, method, headers, and body. Usually this additional data makes it much easier to see what request isn't being mocked. As a bonus, this gives you all the information you need to write a matching handler of your own.

## Make a Lot of Mocks

There's no limit to the number of mocks you can have. Don't be afraid to have a dozen handlers for `https://dynamodb.*.amazonaws.com`, and return `undefined` if you don't want to handle the request. A return value of `undefined` from a handler tells MSW to try the next one in-sequence. For example, I check DynamoDB handlers for a specific table using the following TypeScript:

```ts
export const isGetForTable = async (original: Request, table?: string) => {
  const request = original.clone();
  const body = (await request.json()) as GetItemInput;
  if (table) {
    return body.TableName === table;
  }

  return true;
};
```

DynamoDB operates off of JSON, making it trivial to check the `TableName` for a match. When adding these checks, don't forget to `clone()` the request object! Because MSW uses the built-in `Request` object, you can only read from the request body once, just like when you're using `fetch()`.

Most of your helpers will focus on "is this a `<blank>` command" and "is this a `<blank>` command for resource `<blank>`". You only have to write these once, and then you can reuse them anywhere.

## A Little JSON, A Little XML, A Few Surprises

As you mock, you're going to discover some clients operate on XML while others work with JSON, even though all v3 endpoints support JSON now. Just [roll with it](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-xml-api-responses.html) and follow the AWS API doc's XML responses when required. You'll know when this happens because despite returning JSON, the AWS client will complain about a missing `<` or unhandled `{` in the response.

Some services, like AWS Timestream, make multiple requests. The first request (the one you'd normally associate with an endpoint) just retrieves the *real* endpoint, and the second request goes to this discovered endpoint. When you find these discovery-based services, take advantage of wildcard routes to simplify the network mocks.

Finally, while all this may seem more difficult than the usual mocking pattern, keep with it. The AWS API itself is absurdly stable. Seriously. The SQS API version is tagged November 5, 2012. So once you get these mocks working to your liking, they'll continue to work for the foreseeable future. And if AWS changes something that causes your network requests to change, you'll find out about it immediately.

## Supplement: Known Weird AWS Replies

Because not everything mocked in AWS is obvious, I'll add specific notes about libraries and mocks as I uncover them.

### SQS Uses XML

It's used in almost all infrastructure, but SQS responses are XML. Even worse, you have to include an MD5 digest that is checked by a smithy middleware. The following snippet can create XML success and error responses for the AWS XML API.

```ts
export const sendMessageXMLError = (
  code = "TestingError",
  message = "This is a custom error"
) => /* XML */ `
<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>${code}</Code>
  <Message>${message}</Message>
  <RequestId>7fe4446e-b452-53f7-8f85-181e06f2dd99</RequestId>
</Error>
`;

export const sendMessageXMLResponse = (messageBody: string) => /* XML */ `
<?xml version="1.0" encoding="UTF-8"?>
<SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
    <SendMessageResult>
        <MessageId>374cec7b-d0c8-4a2e-ad0b-67be763cf97e</MessageId>
        <MD5OfMessageBody>${createHash("md5")
          .update(messageBody)
          .digest("hex")}</MD5OfMessageBody>
    </SendMessageResult>
    <ResponseMetadata>
        <RequestId>7fe4446e-b452-53f7-8f85-181e06f2dd99</RequestId>
    </ResponseMetadata>
</SendMessageResponse>
`;

export const createMockSendMessageRespone = (messageBody: string): Response =>
  HttpResponse.xml(sendMessageXMLResponse(messageBody), {
    status: 200,
  });
```

### S3 PutObjectCommand Requires an Empty Body

When putting an object into s3 via `PutObjectCommand`, smithy (the engine underneath the AWS client libraries) expects an HTTP body. A test response needs to include both an empty body and a `Content-Length` header of `0`. This helper makes it easier to create responses for the command.

```ts
type MockResponseBodyOptions = {
  time?: Date;
  etag?: string;
  requestId?: string;
};

export const createMockPutResponse = (
  options?: MockResponseBodyOptions
): Response =>
  HttpResponse.arrayBuffer(new TextEncoder().encode(), {
    status: 200,
    // https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html#API_PutObject_ResponseSyntax
    headers: {
      Etag: options?.etag ?? "1b2cf535f27731c974343645a3985328",
      "Last-Modified": options?.time
        ? options.time.toISOString()
        : new Date().toISOString(),
      "x-amz-request-id":
        options?.requestId ?? "7fe4446e-b452-53f7-8f85-181e06f2dd99",
      "content-length": "0",
    },
  });
```

Link

strong

code

{
    // ...
    "scripts": {
        "test": "NODE_OPTIONS='-r ./test.setup.js' ava"
    }
}


Fence

const handlers: RequestHandler[] = [
  // force inercept of any attempts to call the local AWS Credential Provider
  http.all("http://169.254.169.254/*", () => {
    throw new Error("Attempted to call local AWS Credential Provider");
  }),

  // Provide additional info beyond MSW's default unhandled request error
  http.all("*", async (info) => {
    const request = info.request.clone();

    const debug = {
      url: request.url,
      method: request.method,
      headers: Object.fromEntries(request.headers.entries()),
      body: await request.text(),
    };

    console.error("Unhandled Request");
    console.error(JSON.stringify(debug, null, 2));
  }),
];


export const isGetForTable = async (original: Request, table?: string) => {
  const request = original.clone();
  const body = (await request.json()) as GetItemInput;
  if (table) {
    return body.TableName === table;
  }

  return true;
};


export const sendMessageXMLError = (
  code = "TestingError",
  message = "This is a custom error"
) => /* XML */ `
<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>${code}</Code>
  <Message>${message}</Message>
  <RequestId>7fe4446e-b452-53f7-8f85-181e06f2dd99</RequestId>
</Error>
`;

export const sendMessageXMLResponse = (messageBody: string) => /* XML */ `
<?xml version="1.0" encoding="UTF-8"?>
<SendMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
    <SendMessageResult>
        <MessageId>374cec7b-d0c8-4a2e-ad0b-67be763cf97e</MessageId>
        <MD5OfMessageBody>${createHash("md5")
          .update(messageBody)
          .digest("hex")}</MD5OfMessageBody>
    </SendMessageResult>
    <ResponseMetadata>
        <RequestId>7fe4446e-b452-53f7-8f85-181e06f2dd99</RequestId>
    </ResponseMetadata>
</SendMessageResponse>
`;

export const createMockSendMessageRespone = (messageBody: string): Response =>
  HttpResponse.xml(sendMessageXMLResponse(messageBody), {
    status: 200,
  });


type MockResponseBodyOptions = {
  time?: Date;
  etag?: string;
  requestId?: string;
};

export const createMockPutResponse = (
  options?: MockResponseBodyOptions
): Response =>
  HttpResponse.arrayBuffer(new TextEncoder().encode(), {
    status: 200,
    // https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html#API_PutObject_ResponseSyntax
    headers: {
      Etag: options?.etag ?? "1b2cf535f27731c974343645a3985328",
      "Last-Modified": options?.time
        ? options.time.toISOString()
        : new Date().toISOString(),
      "x-amz-request-id":
        options?.requestId ?? "7fe4446e-b452-53f7-8f85-181e06f2dd99",
      "content-length": "0",
    },
  });


Document

Mocking AWS at the Network Level

```yaml
---
slug: tailwind-css-for-email
published: 2022-10-11T15:00:00-07:00
description: While this won't remove your dependency on table tags, at least you can reuse your Tailwind colors and spacing rules in email templates
changelog: ~
---
```

I can no longer count the hours [Tailwind](https://tailwindcss.com) saves me. I don't really _hate_ the state of CSS-in-JS. I just find it gets in the way of quickly saying "make this blue, put it in a flex box oriented by row, with the items centered." It's a preference thing, and my preference is towards utility class names at this point, letting React components do any reusable composition for me. It's been great, save for one quibble; email is awful. It's not really tailwind's fault, but emails are still table-based, inline styles, and generally everything awful we definitely thought we got rid of in 2002.

What follows isn't complex enough to justify an NPM module, but it is a pattern generally usable for getting Tailwind into your HTML emails. The short version is:

1. A very minimal `tailwind.email.css` is generated using the tailwind CLI. It contains `@tailwind utilities;` and any resets I'd like to do. Be sure you add your email templates to tailwind's config file, otherwise your styles won't be included.
2. We put the generated `tailwind.email.css` through a conversion of `rem` to `px` because it's 2022 and [outlook still does not support the `rem` unit](https://www.caniemail.com/features/css-unit-rem/)
3. [juice](https://github.com/Automattic/juice) inlines the tailwind styles for us where possible, keeping the media queries in the head for email clients that support it
4. A [rehype](https://github.com/rehypejs/rehype) pipeline replaces Tailwind `--tw-` variables with their resolved values

## On Generating the HTML
For what I'm currently doing, emails are built using everyday React and a renderer that calls `ReactDOMServer.renderToStaticMarkup(element)`. The complexities of `tr` and `td` nesting is easily hidden behind some common components, which I chose to name `Row` and `Column` respectively. They're nothing fancy and based on the reality we'll probably never get flexbox in HTML emails in my lifetime.

```tsx
const tableAttributes = {
  role: "presentation",
  cellPadding: 0,
  cellSpacing: 0,
};

interface EmailComponentProps {
  className?: string;
}

export const Row: React.FC<PropsWithChildren<EmailComponentProps>> = ({
  children,
  className,
}) => {
  return (
    <table
      {...tableAttributes}
      {...({ border: "0" } as Record<string, string>)}
      className={cx(className, "w-full")}
    >
      <tr>{children}</tr>
    </table>
  );
};

export const Column: React.FC<PropsWithChildren<EmailComponentProps>> = ({
  children,
  className,
}) => {
  return <td className={className}>{children}</td>;
};
```

Ultimately though, how you generate the HTML is up to you, as long as you have it as a string.

## Tailwind Configuration
To make Tailwind email friendly, we need to disable a bunch of core plugins that don't work well across email clients. A huge shoutout goes to [Can I Email](https://www.caniemail.com). Disabling these plugins is straightforward: add them to your theme's definition, and Tailwind will make sure not to generate CSS for them. Trying to use these CSS classes will just result in useless CSS classes with no style information & can be stripped later.

```js
module.exports = {
  content: [
    // put your email paths here
  ],
  theme: {
    corePlugins: {
      preflight: false,
      backgroundOpacity: false,
      borderOpacity: false,
      boxShadow: false,
      divideOpacity: false,
      placeholderOpacity: false,
      textOpacity: false,
    },
  },
  plugins: [],
};

```

## Converting `rem` to `px`

I wasn't keen on building a completely custom `tailwind.config.js` file, and the `rem` unit is easy enough to convert from as long as you specify a base value in pixels. Tailwind [uses a default base](https://tailwindcss.com/docs/customizing-spacing#default-spacing-scale) of `16px`, so we can convert all of our units via a regular expression.

```ts
const css = originalCSS.replace(
  /([\d.-]+rem)/gi,
  (_, value) => `${parseFloat(value.replace(/rem$/, "")) * 16}px`
);
```

## Applying juice

Our stylesheet is now email friendly, and it's time for [juice](https://github.com/Automattic/juice).

> Given HTML, juice will inline your CSS properties into the style attribute.

Cool. That sounds exactly what we're looking to do. We'll take our HTML plus our px-friendly CSS and ask juice to merge the two together into a final output.

```ts
const juiced = juice(html, { extraCss: css });
```

That's mostly it. Juice takes care of the inline style tags for us where it can, defaulting to a `<style>` tag in the head where it can't (ie media queries), and just generally does the right thing all the time. About the only thing it can't do is fix custom CSS variables, but [there's an issue for CSS variable support](https://github.com/Automattic/juice/issues/312) with a workaround. We'll use code similar to the workaround, but with a tool that's a bit faster than cheerio.

## Resolving CSS Variables

Resolving our CSS variables requires us to make a few assumptions about how email clients behave. First, if a client supports advanced CSS such as `@media` queries, they also support CSS variables. That means the only CSS variables we'll need to care about are those located inline and in the body. Second, we can assume that Tailwind isn't going to generate any weird syntax. CSS variables will be in the form of `--declaration: value`, and usage will be in the form of `var(--declaration)` with an optional fallback.

Replacing the variables will require two passes. On the first pass, we'll capture all our CSS variables assigned to the inline style, and on the second, we'll replace our `var()` statements with the resolved values. Using [`rehype`](https://github.com/rehypejs/rehype) and [`rehype-rewrite`](https://github.com/jaywcjlove/rehype-rewrite) makes this a straightforward task.

```ts
const variableDefRegex = /(--[a-zA-Z0-9-_]+)\s*:\s(.+?);/g;
const variableUsageRegex = /var$(\s*--[a-zA-Z0-9-_]+\s*)(?:$|,\s*(.*)\))/;
const hyped = rehype()
  .use(rehypeRewrite, {
    rewrite: (node) => {
      if (node.type !== "element") {
        return node;
      }
      const resolveVariables = (s: string): string => {

        // pass 1: pull definitions
        const defs = new Map<string, string>();
        let withoutDefs = s.replace(
          variableDefRegex,
          (_, def: string, value: string) => {
            defs.set(def.trim(), value.trim());
            return "";
          }
        );

        // pass 2: replace variables (maxCycles prevents Terrible Things from happening)
        let maxCycles = 1000;
        while (withoutDefs.match(variableUsageRegex)) {
          maxCycles--;
          if (maxCycles <= 0) {
            throw new Error("Max Cycles for replacement exceeded");
          }
          withoutDefs = withoutDefs.replace(
            variableUsageRegex,
            (_, def: string, fallback: string) => {
              const d = def.trim();
              if (defs.has(d)) {
                return defs.get(d) ?? "";
              }
              return (fallback ?? "").trim();
            }
          );
        }

        // return clean result
        return withoutDefs;
      };
      node.properties = {
        ...node.properties,
        style: resolveVariables(`${node.properties?.style ?? ""}`),
      };
    },
  })
  .use(stringify)
  .processSync(juiced)
  .toString();
```

## Wrapping Up

At this point, I'm pretty happy with where the code is. The generated tailwind output from the email.css is easy to understand and manage since it's just vanilla tailwind. Juice is doing what it does best, and the only manual adjustments in the code are converting the `rem` to `px` and fixing of inline CSS variables. That's a really small amount of maintained code relative to an email specific library and build system in exchange for the full power of Tailwind. Emails still suck, but hopefully they'll suck a little less. With the Libraries in place, we put together the Taskless Usage in about 20 minutes.

![image](https://user-images.githubusercontent.com/1795/193527500-a0ada56c-272b-4809-b886-8f112312ef5e.png)

And the associated code:

```ts
/* mailwindCss.ts */
import { readFile } from "node:fs/promises";
import juice from "juice";
import { rehype } from "rehype";
import rehypeRewrite from "rehype-rewrite";
import stringify from "rehype-stringify";

/** Cache of compiled tailwind files */
const twcCache = new Map<string, Promise<string>>();

/**
 * Originally based on The MailChimp Reset from Fabio Carneiro, MailChimp User Experience Design
 * More info and templates on Github: https://github.com/mailchimp/Email-Blueprints
 * http://www.mailchimp.com &amp; http://www.fabio-carneiro.com
 * These styles are non-inline; they impact UI added by email clients
 * By line:
 * (1) Force Outlook to provide a "view in browser" message
 * (2) Force Hotmail to display emails at full width
 * (3) Force Hotmail to display normal line spacing
 * (4) Prevent WebKit and Windows mobile changing default text sizes
 * (5) Remove spacing between tables in Outlook 2007 and up
 * (6) Remove table borders on MSO 07+ http://www.campaignmonitor.com/blog/post/3392/1px-borders-padding-on-table-cells-in-outlook-07/
 * (7) Specify bicubic resampling for MSO on img objects
 * (8) Media Query block - Pretty phone numbers in email: http://www.campaignmonitor.com/blog/post/3571/using-phone-numbers-in-html-email
 * (9) Media Query block - same as above, but for tablet sized devices
 */
const universalStyles = /*css*/ `
  #outlook a{ padding:0; }
  .ReadMsgBody{ width:100%; } .ExternalClass{ width:100%; }
  .ExternalClass, .ExternalClass p, .ExternalClass span, .ExternalClass font, .ExternalClass td, .ExternalClass div { line-height: 100%; }
  body, table, td, p, a, li, blockquote{ -webkit-text-size-adjust:100%; -ms-text-size-adjust:100%; }
  table, td{ mso-table-lspace:0pt; mso-table-rspace:0pt; }
  table td {border-collapse: collapse;}
  img{-ms-interpolation-mode: bicubic;}
  @media only screen and (max-device-width: 480px) {
    a[href^="tel"],
    a[href^="sms"] {
      text-decoration: default !important;
      pointer-events: auto !important;
      cursor: default !important;
    }
  }
  @media only screen and (min-device-width: 768px) and (max-device-width: 1024px) {
    a[href^="tel"],
    a[href^="sms"] {
      text-decoration: default !important;
      pointer-events: auto !important;
      cursor: default !important;
    }
  }
`;

/**
 * Inline reset styles
 * These set common defaults for a consistent UI. Their comments will be automatically stripped by juice
 */
const resetStyles = /*css*/ `
  /* default margin/padding, box sizing*/
  * { margin: 0; padding: 0; box-sizing: border-box; }
  /* height 100% all the way down */
  table, tr, td {height: 100%;}
  /* img behavior - bicubic ms resizing, no border, fix space gap on gmail/hotmail */
  img { outline: none; text-decoration: none; -ms-interpolation-mode: bicubic; display: block; }
  a img { border: none; }
  /* mso 07, 10 table spacing fix http://www.campaignmonitor.com/blog/post/3694/removing-spacing-from-around-tables-in-outlook-2007-and-2010 */
  table { border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt;}
  /* tel/sms links are unstyled by default */
  a[href^="tel"], a[href^="sms"] { text-decoration: none; pointer-events: none; cursor: default; }
`;

const variableDefRegex = /(--[a-zA-Z0-9-_]+)\s*:\s(.+?);/g;
const variableUsageRegex = /var$(\s*--[a-zA-Z0-9-_]+\s*)(?:$|,\s*(.*)\))/;

/** Juices the styles, and then resolves CSS variables and additional HTML adjustements via rehype */
const inlineStyles = (html: string, css?: string) => {
  const juiced = juice(html, { extraCss: css });
  const hyped = rehype()
    .use(rehypeRewrite, {
      rewrite: (node) => {
        if (node.type !== "element") {
          return node;
        }

        // inline styles into the <head>
        if (node.tagName === "head") {
          node.children = [
            ...node.children,
            {
              type: "element",
              tagName: "style",
              children: [{ type: "text", value: universalStyles }],
            },
          ];
        }

        const resolveVariables = (s: string): string => {
          // pass 1: pull definitions
          const defs = new Map<string, string>();
          let withoutDefs = s.replace(
            variableDefRegex,
            (_, def: string, value: string) => {
              defs.set(def.trim(), value.trim());
              return "";
            }
          );

          // pass 2: replace variables
          let maxCycles = 1000;
          while (withoutDefs.match(variableUsageRegex)) {
            maxCycles--;
            if (maxCycles <= 0) {
              throw new Error("Max Cycles for replacement exceeded");
            }
            withoutDefs = withoutDefs.replace(
              variableUsageRegex,
              (_, def: string, fallback: string) => {
                const d = def.trim();
                if (defs.has(d)) {
                  return defs.get(d) ?? "";
                }
                return (fallback ?? "").trim();
              }
            );
          }

          // return clean result
          return withoutDefs;
        };
        node.properties = {
          ...node.properties,
          style: resolveVariables(`${node.properties?.style ?? ""}`),
        };
      },
    })
    .use(stringify)
    .processSync(juiced)
    .toString();
  return hyped;
};

interface MailwindOptions {
  /** A path to your tailwind.css file, optimized for email */
  tailwindCss: string;
  /** The base px value for 1rem, defaults to 16px */
  basePx?: number;
  /** Set to `false` to disable extended resets */
  reset?: boolean;
}

const mailwindCss = async (email: string, options: MailwindOptions) => {
  const basePx = options?.basePx ?? 16;

  // cache promise for performance in serverless environments
  if (
    !twcCache.has(options.tailwindCss) ||
    process.env.NODE_ENV === "development"
  ) {
    // console.log("loading styles");
    const p = new Promise<string>((resolve, reject) => {
      readFile(options.tailwindCss)
        .then((buf) => {
          const s = buf.toString();
          // rem to px
          const pxed = s.replace(
            /([\d.-]+rem)/gi,
            (_, value) => `${parseFloat(value.replace(/rem$/, "")) * basePx}px`
          );
          resolve(pxed);
        })
        .catch((reason) => reject(reason));
    });
    twcCache.set(options.tailwindCss, p);
  }

  const s = await twcCache.get(options.tailwindCss);
  if (typeof s === "undefined") {
    throw new Error(`Could not load tailwind css from ${options.tailwindCss}`);
  }

  return inlineStyles(
    email,
    options.reset === false ? s : [resetStyles, s].join("\n")
  );
};

export default mailwindCss;
```


While this won't remove your dependency on table tags, at least you can reuse your Tailwind colors and spacing rules in email templates

module.exports = {
  content: [
    // put your email paths here
  ],
  theme: {
    corePlugins: {
      preflight: false,
      backgroundOpacity: false,
      borderOpacity: false,
      boxShadow: false,
      divideOpacity: false,
      placeholderOpacity: false,
      textOpacity: false,
    },
  },
  plugins: [],
};



const css = originalCSS.replace(
  /([\d.-]+rem)/gi,
  (_, value) => `${parseFloat(value.replace(/rem$/, "")) * 16}px`
);


blockquote

const juiced = juice(html, { extraCss: css });


/* mailwindCss.ts */
import { readFile } from "node:fs/promises";
import juice from "juice";
import { rehype } from "rehype";
import rehypeRewrite from "rehype-rewrite";
import stringify from "rehype-stringify";

/** Cache of compiled tailwind files */
const twcCache = new Map<string, Promise<string>>();

/**
 * Originally based on The MailChimp Reset from Fabio Carneiro, MailChimp User Experience Design
 * More info and templates on Github: https://github.com/mailchimp/Email-Blueprints
 * http://www.mailchimp.com &amp; http://www.fabio-carneiro.com
 * These styles are non-inline; they impact UI added by email clients
 * By line:
 * (1) Force Outlook to provide a "view in browser" message
 * (2) Force Hotmail to display emails at full width
 * (3) Force Hotmail to display normal line spacing
 * (4) Prevent WebKit and Windows mobile changing default text sizes
 * (5) Remove spacing between tables in Outlook 2007 and up
 * (6) Remove table borders on MSO 07+ http://www.campaignmonitor.com/blog/post/3392/1px-borders-padding-on-table-cells-in-outlook-07/
 * (7) Specify bicubic resampling for MSO on img objects
 * (8) Media Query block - Pretty phone numbers in email: http://www.campaignmonitor.com/blog/post/3571/using-phone-numbers-in-html-email
 * (9) Media Query block - same as above, but for tablet sized devices
 */
const universalStyles = /*css*/ `
  #outlook a{ padding:0; }
  .ReadMsgBody{ width:100%; } .ExternalClass{ width:100%; }
  .ExternalClass, .ExternalClass p, .ExternalClass span, .ExternalClass font, .ExternalClass td, .ExternalClass div { line-height: 100%; }
  body, table, td, p, a, li, blockquote{ -webkit-text-size-adjust:100%; -ms-text-size-adjust:100%; }
  table, td{ mso-table-lspace:0pt; mso-table-rspace:0pt; }
  table td {border-collapse: collapse;}
  img{-ms-interpolation-mode: bicubic;}
  @media only screen and (max-device-width: 480px) {
    a[href^="tel"],
    a[href^="sms"] {
      text-decoration: default !important;
      pointer-events: auto !important;
      cursor: default !important;
    }
  }
  @media only screen and (min-device-width: 768px) and (max-device-width: 1024px) {
    a[href^="tel"],
    a[href^="sms"] {
      text-decoration: default !important;
      pointer-events: auto !important;
      cursor: default !important;
    }
  }
`;

/**
 * Inline reset styles
 * These set common defaults for a consistent UI. Their comments will be automatically stripped by juice
 */
const resetStyles = /*css*/ `
  /* default margin/padding, box sizing*/
  * { margin: 0; padding: 0; box-sizing: border-box; }
  /* height 100% all the way down */
  table, tr, td {height: 100%;}
  /* img behavior - bicubic ms resizing, no border, fix space gap on gmail/hotmail */
  img { outline: none; text-decoration: none; -ms-interpolation-mode: bicubic; display: block; }
  a img { border: none; }
  /* mso 07, 10 table spacing fix http://www.campaignmonitor.com/blog/post/3694/removing-spacing-from-around-tables-in-outlook-2007-and-2010 */
  table { border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt;}
  /* tel/sms links are unstyled by default */
  a[href^="tel"], a[href^="sms"] { text-decoration: none; pointer-events: none; cursor: default; }
`;

const variableDefRegex = /(--[a-zA-Z0-9-_]+)\s*:\s(.+?);/g;
const variableUsageRegex = /var$(\s*--[a-zA-Z0-9-_]+\s*)(?:$|,\s*(.*)\))/;

/** Juices the styles, and then resolves CSS variables and additional HTML adjustements via rehype */
const inlineStyles = (html: string, css?: string) => {
  const juiced = juice(html, { extraCss: css });
  const hyped = rehype()
    .use(rehypeRewrite, {
      rewrite: (node) => {
        if (node.type !== "element") {
          return node;
        }

        // inline styles into the <head>
        if (node.tagName === "head") {
          node.children = [
            ...node.children,
            {
              type: "element",
              tagName: "style",
              children: [{ type: "text", value: universalStyles }],
            },
          ];
        }

        const resolveVariables = (s: string): string => {
          // pass 1: pull definitions
          const defs = new Map<string, string>();
          let withoutDefs = s.replace(
            variableDefRegex,
            (_, def: string, value: string) => {
              defs.set(def.trim(), value.trim());
              return "";
            }
          );

          // pass 2: replace variables
          let maxCycles = 1000;
          while (withoutDefs.match(variableUsageRegex)) {
            maxCycles--;
            if (maxCycles <= 0) {
              throw new Error("Max Cycles for replacement exceeded");
            }
            withoutDefs = withoutDefs.replace(
              variableUsageRegex,
              (_, def: string, fallback: string) => {
                const d = def.trim();
                if (defs.has(d)) {
                  return defs.get(d) ?? "";
                }
                return (fallback ?? "").trim();
              }
            );
          }

          // return clean result
          return withoutDefs;
        };
        node.properties = {
          ...node.properties,
          style: resolveVariables(`${node.properties?.style ?? ""}`),
        };
      },
    })
    .use(stringify)
    .processSync(juiced)
    .toString();
  return hyped;
};

interface MailwindOptions {
  /** A path to your tailwind.css file, optimized for email */
  tailwindCss: string;
  /** The base px value for 1rem, defaults to 16px */
  basePx?: number;
  /** Set to `false` to disable extended resets */
  reset?: boolean;
}

const mailwindCss = async (email: string, options: MailwindOptions) => {
  const basePx = options?.basePx ?? 16;

  // cache promise for performance in serverless environments
  if (
    !twcCache.has(options.tailwindCss) ||
    process.env.NODE_ENV === "development"
  ) {
    // console.log("loading styles");
    const p = new Promise<string>((resolve, reject) => {
      readFile(options.tailwindCss)
        .then((buf) => {
          const s = buf.toString();
          // rem to px
          const pxed = s.replace(
            /([\d.-]+rem)/gi,
            (_, value) => `${parseFloat(value.replace(/rem$/, "")) * basePx}px`
          );
          resolve(pxed);
        })
        .catch((reason) => reject(reason));
    });
    twcCache.set(options.tailwindCss, p);
  }

  const s = await twcCache.get(options.tailwindCss);
  if (typeof s === "undefined") {
    throw new Error(`Could not load tailwind css from ${options.tailwindCss}`);
  }

  return inlineStyles(
    email,
    options.reset === false ? s : [resetStyles, s].join("\n")
  );
};

export default mailwindCss;


Tailwind CSS for Email

```yaml
---
slug: expo-community-flipper-retired
published: 2023-08-04T18:19:00-07:00
description: Thoughts on "completing" an open source project
excerpt: That moment when you're comfortable putting a temporary project to rest.
changelog: ~
---
```

With [the release of Expo 48](https://blog.expo.dev/expo-sdk-48-ccb8302e231), `expo-community-flipper` (gh)[https://github.com/jakobo/expo-community-flipper] (npm)[https://www.npmjs.com/package/expo-community-flipper] can finally declare its purpose fulfilled. A year ago, I was trying to debug redux inside of the Chrome Debugger and nearly lost my mind. It wasn't just a bad experience. It was an **awful** experience, defined by `console.log` and prayers.

There was [react-native debugger](https://github.com/jhen0409/react-native-debugger) and [reactotron](https://github.com/infinitered/reactotron), but nothing that worked for me specifically. My setup was **weird** but not unlikely. WSL2, Windows 11, and the iOS app running via Expo's custom dev app. Only one thing really worked with this setup, and that was [Flipper](https://fbflipper.com/). Plus, I could finally see FPS information for animations as an added bonus (more on that later).

## Expo & Flipper in a Graph

![Download statistics for expo-community-flipper](https://user-images.githubusercontent.com/1795/222308501-930d3ee3-1121-4895-801d-0c3a544d7f88.png)

One of the problems with using Flipper was that it required editing the Podfile. This is all well and good if you're using react-native on a mac and love running XCode yourself. But if you're that weird-dev-guy with WSL 2, running XCode is ~~impossible~~ legally troublesome at best. As a lucky coincidence, Expo pushed out SDK 43 and began documenting their config plugin system: scriptable JavaScript that could edit & change build files as part of a remote CI or build pipeline.

And so, starting with SDK 43, I released the [first version of expo-community-flipper on npm](https://www.npmjs.com/package/expo-community-flipper/v/43.0.0) into the wild. With every subsequent SDK, I'd check the [expo base template](https://github.com/expo/expo/tree/main/templates/expo-template-bare-minimum/ios) and the [React Native Upgrade Helper](https://react-native-community.github.io/upgrade-helper/) and focus on changes to the `Podfile`.

It was a scramble. It wasn't always obvious _what_ changed. Sometimes a random `AppDelegate.m` was the culprit, other times it was the ruby scripts to include Flipper upstream in the React Native libraries. It got better though with every release.

* [I wrote a post for the Expo team](https://blog.expo.dev/developing-react-native-with-expo-and-flipper-8c426bdf995a) to go along with my WSL 2 article I did earlier for them
* [Alexandre Moureaux hooked performance monitoring in](https://www.youtube.com/watch?v=Yl6gj0ZLB3Y) and gave a talk
* [Abdulrahman wrote an article](https://medium.com/@abdulrahmanhashem/8-ways-to-debug-your-react-native-app-with-expo-the-comprehensive-guide-538655cf1684)

Debugging was finally _easier_. It would just be nice to get out of reaction-mode when new SDKs were imminent.

## Enter the Expo Team and Merging Up

SDK 47 marked the beginning of the end in a positive way. There were changes coming that would break `expo-community-flipper`, and the expo team wanted to be proactive about the changeset. This was the first time in the project's history that `expo-community-flipper` was going to be SDK ready on launch. This proactive outreach got the wheels turning though. What if expo just did the "Flipper bits" itself? They've got a much better testing suite, there are already plugins for modifying the `Podfile`, and then Flipper could get the attention it deserved.

Ideally, SDK 47 would be the last time we'd have to do this npm release dance. After talking with the Expo team, we [launched a plan to merge `expo-community-flipper` upstream](https://github.com/jakobo/expo-community-flipper/issues/41) into the `build-properties` plugin. [It landed in January](https://github.com/expo/expo/pull/20890) and Flipper support became an official part of the `expo-config-build-properties` plugin in SDK 48.

This section is pretty uneventful because the Expo team is just that good. I worked with [Kudo Chien](https://github.com/Kudo) for most of the PR and integration, and Expo had the infrastructure to absorb the changeset without issue. I was honestly impressed at how smooth everything went. I also like the "Why" and "How" sections they encourage on PRs for explaining their changes.

## Ending `expo-community-flipper`

Finally came the goodbyes. A README change, a release on SDK 48 with a `peerDependency` that keeps it from running against newer Expo versions, and an `npm deprecate`. And it was done. With Expo SDK 49 released and deprecating 46, and SDK 50 on the horizon, `expo-community-flipper` will be finally be archived and the project complete.

It's surreal. It feels good. Not many open source projects get a happily ever after.

Thoughts on "completing" an open source project

The End of expo-community-flipper

```yaml
---
slug: transactions-part-1
published: 2006-04-13
description: Gaia's original "transaction manager" explained
excerpt: ~
changelog:
  - on: 2020-02-23
    note: This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal). As a result, many links in this piece were left pointing to archive.org destinations.
---
```

Everything is on track for SXSWi this year, and excited isn't even a good word for how I feel right now. It's 4 am [Ed. Now it is noon the following day…] and I'm downright giddy. Last year I blew my first chance to go because I didn't move fast enough. This year, I jumped on travel planning much quicker and managed to grab a room at the Hilton Austin. I'm in on the 10th and out on the 14th, and if I didn't have some larger projects, I'd be staying much longer. To those going, I'll see you there (just look for the kid who looks like an excited lost puppy). However, if you notice the title, this isn't about SXSWi entirely, but instead is focused much more on some meaty PHP things: transactions. For the new bank code at Gaia, there needed to be a way to move items and gold about the site through a standardized (and [5 nines" reliability) system. Add to that full logging in the event of failure, and the ability to "transact" just about any two things on the site, and you get the foundation of the transaction system. This is part one of a multi-part article which will touch on the surface and theories used in the transactions framework. (Part 2 will focus more or architecture.)

* **Part 1: Theory**
* [Part 2, Transaction Framework](/thunked/transactions-part-2)

When the project first began, transactions were an odd concept. Not like database transactions, but a sequence of events that were atomically linked and reversible. Examples included purchasing from the Gaia Store (give the system gold, the system gives an item), making a trade (place an item into an "escrow" neutral zone), auctioning off an item (again, an escrow style neutral zone), purchasing non-pixels such as a Trading Pass or a Guild Purchase where the result is a database row but no items or gold, and many other things that went beyond the basic "remove gold, add item" mentality. This meant it was time for some research into complex data analysis models. Enter Martin Fowler, whose books I had read back in college, but like most college students made a point of forgetting. Thankfully, John had Analysis Patterns which was all we needed to get underway. I'm going to digress for a moment to say Fowler is a very intelligent man, but I remember why I couldn't remember what he wrote- Fowler has a hard time communicating why a pattern works the way it does in a way that connected with me. If you're someone who needs to know the "why", you might have to suck it up like I had to. After going cover to cover, it appeared there was a very basic system underneath even the most complex banking systems. To abridge the text, we started referring to it as the Conceptual Account Thingy (or CAT for short).

## Accounts, Entries, and All That Jazz
Keeping with our simplified version of things, an Account was described as nothing more than a container. This container could hold things, and those things belong to some entity. As an added bonus, [Gaia's "DAO"](http://72squared.com/index.php/module/Articles/action/Display/id/37) information was also stored with the Account in order to provide a means to get out and save data when needed. Accounts were then abstracted and an Account class was created for the "things" on Gaia that were owned by "someone". The initial list included:

* Gold (owned by user)
* Inventory (owned by user)
* Trades (owned by the trade row, with two users participating)

The third one is very interesting because a Trade became what was called a "Summary Account" in that it contained two Accounts inside of it, a left half and a right half. The left half was owned by the trade row and by the left user, while the right half was owned by the trade row and the right user. Because of this, the only entity that owns the entire trade is the trade row itself. When retrieving Accounts, they are always fetched by their owner which is often a primary key. This ensures speed of performance during the Account loading process. With a means to load and save the data, the next step is to manipulate it.

According to Fowler, the Entry class is nothing more than an entry that will be logged to an Account. Think of recording a check in the checkbook's ledger. This process is formally called Posting an entry, but for practical purposes, entries are applied to an Account, and then the changed Account is saved back to the database. To soften the load on PHP, the entries became lightweight containers of metadata which provided little more than a pretty interface for method calls. A simplified version of an Entry looked like: (note, "Container" is nothing more than an abstracted hash table in order to provide useful namespacing functionality. In this case, the `datatypecontainer` enforces typecasting on items going in and coming out of it.)

```php
<?PHP
/**
* Required Include Files
*/
require_once(DIR_CLASSES . 'container/datatypecontainer.php');

class TransactionEntry
{
    function TransactionEntry()
    {
        $this->Container = new DataTypeContainer();
    }
    function importData($data) {}

    function exportData($preserveId = true) {}

    function setEntryId($v) {}
    function getEntryId() {}

    function setTransactionId($v) {}
    function getTransactionId() {}

    function setEntityId($v) {}
    function getEntityId() {}

    function setAccountType($v) {}
    function getAccountType() {}

    function setObjectId($v) {}
    function getObjectId() {}

    function setAttribute($v) {}
    function getAttribute() {}

    function setAsAdd() {}
    function setAsSubtract() {}
    function setPolarity($v) {}
    function getPolarity() {}

    function setTimestamp($v) {}
    function getTimestamp() {}

    function setStatus($v) {}
    function getStatus() {}

    function cloneInverse()
    {
        $obj = new TransactionEntry();
        $data = $this-&gt;exportData(false);
        $data["polarity"] = $data["polarity"] * -1;
        $obj-&gt;importData($data);
        return $obj;
    }
}

?>
```

I left the `cloneInverse` statement because it was of particular note. From above, it was said that everything done to an Account needs to be able to be undone. The secret to this is being able to "reverse" all of the entries that have been applied to a transaction, and then reapply them. The net result of this will always be the Account's original state. In extensions of this Entry object, decoration aliases are used such as `setUserId()` in order to simplify the application programmers' work. With entries, and things to apply those entries to, the next step is to set up the rules.

## Posting Rules
When the transaction manager hands an entry to an Account, it is the Account's responsibility to take that entry and apply it. For different Accounts, applying an entry means different things. Inventory Accounts (owned by a user) are by far the hardest with their need to add/remove binary strings. An Account can always look at the class of an Entry object if that Account supports multiple types of entries. At present, since the inventories on Gaia (housing, main, game inventory, etc) are separated, there is no need to develop sophisticated entry handling. Fowler gets into extreme detail about Posting Rules and covers more than is needed by this system. However, the basics of how to apply a rule and what happens when applying an entry to an Account need to still be established. It was agreed upon that should a posting rule fail, it will fail all future objects in the posting rule change, and triggering a rollback of the transaction by taking the `cloneInverse()` of each Entry and reapplying them.

![image](https://user-images.githubusercontent.com/1795/146686016-cfbac26c-f5a8-4af8-99c9-f263450b608b.png)

Every `Account` then has a public `postEntry()` method, which takes an `Entry` object as an argument. Using the generic accessor functions from the base `TransactionEntry` object, the `Account` can apply the Object ID, Attribute, and Polarity to their entry. Each call to `postEntry()` returns `TRUE` on rule posting success. All that remains is to tackle the fact that the web (and the Accounts) are constantly in use. Collisions can occur when a second transaction manager is started while the first one is still in process. Since the first transaction hasn't saved, the second process has dirty stale old data.

Fowler presents two methods of avoiding collision, both which are worth touching on briefly. Mileage varies with either one.

## Posting Rules With "Catchup"
Of the two methods Fowler discussed, journaling is by far the most widely employed. To draw a real-life analogy, it functions much like a checkbook register. At any given point, a quick glance through the register can tell someone what the "balance" is, including all transactions which haven't cleared. Much like processing checks out of order, an Account can be measured by taking its present state, and then applying all deduction Entry objects which have not cleared yet. This gives the transaction manager the net low for the account, making it possible for the system to check for insufficient funds, missing items, and incomplete database rows. In a set of linear steps in PHP/MySQL, you might get something like the following:

1. Process 1 writes what it wants to do (status uncleared) to the DB
2. Process 1 gets an insert ID as a result of this
3. Process 1 loads the Account
4. Process 1 loads all deducting transactions up to its insert ID on top of the Account
5. Process 2 can start, and at any point after #<span/>1, its Net Account will include all of Process 1's unresolved deductions.

While this is a very reliable system, the need to constantly bring an Account to current isn't a very cheap process query wise. Also, every system needs to implement this transaction system so that a blind write to someone's inventory isn't destroyed in the process. Since the Vending system, Shopping system, Item Granting System (used in games) all would not start equipped with this transaction manager, it was not meant to be. If you're starting from scratch, this is certainly the way to go. If not, well, you can always improvise.

## Improvising with innoDB Locking
The problem was, as stated above, the asynchronous nature of Account objects. Lacking the ability to journal the transactions, the other solution to an Asynchronous problem is to break it. By redefining the Accounts to be as small a data set as possible and introducing a simple retry. The most likely collision on the site is the Gold Account, which would require a user to earn gold on the site within the same timeframe as say, completing a Marketplace Transaction. Marketplace can see the lock, wait a split second, and try again. The resulting failing collision rate is so low that it's considered acceptable to have a user retry in the event of a complete lockout.

I've made mention a few times to locking, and this is done with two tools: innoDB and the MySQL `SELECT...FOR UPDATE` syntax. We've made liberal use of the innoDB storage engine for it's transaction support, and according to MySQL, using the `SELECT...FOR UPDATE` syntax places a good lock on the innoDB row, letting the process finish what it was doing. If during the select phase some database row it needs is locked, it reseeds at a small random interval and performs its work on a retry. When complete, the transaction is committed and the changes recorded.

For spanning multiple databases with a single transaction, John developed a DB spanning object for the DAO he developed. It chains commits to fire off in sequence, effectively minimizing the failing window gap. Half commits are still a problem should for example, all the servers magically lose power and they are mid-commit. Unfortunately, this is a failing of the improvisation that can only be solved with a true multi-phase commit. In production testing, it's failure rate is wholly dependent on the hardware's reliability. While this has a low failing rate, it is substantial enough to warrant exploring extensions to the DAO's DB-spanning system.

## Conclusions

The system is currently in testing with a very good success rate. There are a few tweaks needed to optimize the system of Account and Entry objects in order to reduce weight when including them for processing. I'd also like to better explore the DAO's integration into the Account object- the current method alters the conceptual DAO behavior in order to solve a problem where more than one Account object may be tied to the same database row. More on that next time though.

This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal). As a result, many links in this piece were left pointing to archive.org destinations.

Gaia's original "transaction manager" explained

<?PHP
/**
* Required Include Files
*/
require_once(DIR_CLASSES . 'container/datatypecontainer.php');

class TransactionEntry
{
    function TransactionEntry()
    {
        $this->Container = new DataTypeContainer();
    }
    function importData($data) {}

    function exportData($preserveId = true) {}

    function setEntryId($v) {}
    function getEntryId() {}

    function setTransactionId($v) {}
    function getTransactionId() {}

    function setEntityId($v) {}
    function getEntityId() {}

    function setAccountType($v) {}
    function getAccountType() {}

    function setObjectId($v) {}
    function getObjectId() {}

    function setAttribute($v) {}
    function getAttribute() {}

    function setAsAdd() {}
    function setAsSubtract() {}
    function setPolarity($v) {}
    function getPolarity() {}

    function setTimestamp($v) {}
    function getTimestamp() {}

    function setStatus($v) {}
    function getStatus() {}

    function cloneInverse()
    {
        $obj = new TransactionEntry();
        $data = $this-&gt;exportData(false);
        $data["polarity"] = $data["polarity"] * -1;
        $obj-&gt;importData($data);
        return $obj;
    }
}

?>


Transacting Items on Gaia Online

```yaml
---
slug: javascript-threading
published: 2007-11-08T00:00:00-08:00
description: Not Everything Adheres to JavaScript's Semi-Single-Threaded Model
changelog:
  - on: 2020-02-23
    note: This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal). As a result, many links in this piece were left pointing to archive.org destinations.
  - on: 2020-02-24
    note: While I couldn't find the test page, I'm happy to report that this bug no longer exists in Firefox and IE on the Mac is also no longer a concern for JS engineers.
---
```

There's a lot of digging required to figure out if JavaScript is single or multi-threaded. Thanks to the work Dan Simard in June, [a final answer on JavaScript threads was reached](https://web.archive.org/web/20071118102059/http://www.javascriptkata.com/2007/06/12/ajax-javascript-and-threads-the-final-truth/). The only major problem in this was that the comments by a user to Simard's post indicated that `alert()` caused Firefox's execution chain to change. The result was while the prompt was on screen, `setTimeout()` and other assorted callback functions were free to resolve.

The problem is made worse with several `setTimeout()` calls. As a single thread, JavaScript maintains some sort of event stack, where all of the callbacks and timeouts pile up, waiting for their turn. Once the event stack is out of events, the timers start again to count down to execution. If you fire one call with a `setTimeout()` and a timeout value of 2 seconds, and one more second of code execution follows, there will still be 2 seconds on the timer since it has not had a chance to run.

```js
var foo_count = 0;
var bar_count = 0;
var foo = function () {
  ++foo_count;
  window.setTimeout(bar, 5);
};
var bar = function () {
  ++bar_count;
  alert("foo=" + foo_count + ", bar=" + bar_count);
};

// triggers
window.setTimeout(foo, 10);
window.setTimeout(bar, 20);
// end triggers
```

In a perfectly timed environment (running just this code), `foo` will execute, followed by `bar`, followed by `foo`'s call to `bar`. In reality, the millisecond calibration for this is iffy. The output comes out as `foo=1`, `bar=2` then `foo=1`, `bar=1`. However, simply looking at the timing values, our alert calls should be reversed. We've entered the world of volatile variables and shared space without even knowing (and probably not even wanting). Under the hood, it appears that making a call to `alert()`, `prompt()`, `confirm()`, or any other window level call pauses the current thread's execution, letting any pending `setTimeout()` calls resolve. When the execution returns from the window event, variables that were scoped one block higher could have changed without the knowledge of the current execution block.

A more practical example of this can be found on the thread safe ~~JS Example Page~~. As of this writing, IE and Opera execute things properly in order, waiting for the prompt() to finish before continuing the current code block, and after completion, letting the pending `setTimeout()` event fire. In Firefox and Safari, during the `prompt()` call, the `setTimeout()` executes, overwriting the value of bar. [Ed-- unfortunately, the test page is currently MIA. If you find it, I'd love to relink it!]

## Protecting a Volatile Bar
As mentioned before, when the browser's JS Interpreter hands off to the `alert()`, it begins resolving any waiting `setTimeout()` calls that in in the event queue. In our example, another call to `bar()` is waiting which also increments `bar_count`. By the time we alert `bar_count`, something has gone and changed it. This seems to also hold true for `prompt()` and `confirm()` in Firefox and a few other browsers.

Imagine instead of just our primitive `window.setTimeout()` we were using an AJAX callback which waited for a DOM node to be available. The costs of this shared variable space being violated is a bit more costly. In many other languages, there are Mutexes for the synchronized calls, and the volatile declarative to avoid problems. Because the implementation of ECMAScript is unique to each browser, there needs to be a consistent way to secure data from tampering during the application's critical moments.

At our disposal...

- `new Object()` is atomic (can't be interrupted by browser events)
- `return (new Object() === new Object());` is always false as each new object is given a unique ID by the system. We need a way to atomically create an ID for a fake thread, and the Object's deep equality can do just that given it is atomic.
- `Array.push()` is atomic (can't be interrupted by browser events) as long as we are using the native implementation. Even if two objects push onto an array in a race condition, only one will hold the coveted index of `0`.
- `(function() { new Object(); })();` is garbage collected properly
- block scope level variables cannot be changed by outside factors

That doesn't leave us with much. [Ming made a post on JS Mutexes](https://web.archive.org/web/20071118102059/http://my2iu.blogspot.com/2006/10/javascript-mutexes-2.html) though that gave me an idea. The biggest concern from Ming's entries was that in Internet Explorer, events such as onload can interrupt a process, creating multiple events at the same time. To get around this, we would need a method which the browser cannot interrupt. This is where the native `Array.push()` comes in handy. For some reason, IE treats this call as non-interruptible, letting us have a queue process that ensures element `0` is the first lock requester. If we have two atomic commands, one to uniquely identify a lock request and another to enqueue the lock, then we have everything we need for a very basic spinlock implementation. Even if a second event interrupts our two statements, the pushing onto the array guarantees only one of the two requests will be given rights to the lock. We'll know which function got the lock thanks to our deep quality `===` of objects.

Spinlock design implies a "busy wait" cycle for getting the lock, meaning each individual thread usually sits in the equivalent of `while(1) {}`. Because we don't have threads and instead have the event queue, a while loop would seize the system. To get around this, we can use the same `setTimeout()` call that was the source of our problem to "sleep" an event until the lock resource is available. The result is a very lazy locking method that requires the application to care about the lock (but a lock that can be used in many different ways). Our pseudo code looks like:

```
sub process
    lock = get lock
    if not lock owner
        retry sub process in N ms (sleep)
        return

    lock was obtained
    perform critical code
    release lock

    non critical segment here
end process
```

Return to the ~~thread safe test page~~ and try the protected version. Across all browsers, the lock is held until the our asynchronous events properly completed.

## Beyond Locking Down Variables
A locking system like this can be extended to deal with more than just the JavaScript Variables. In the case of lazy loading JavaScript, it is possible to be using the same utility to load several "sets" of JavaScript at the same time. To avoid keeping the `document.head` open (and risking Operation Aborted problems), we use the same locking strategy to keep a DOM node from getting written to until Internet Explorer reports it as closed, such as an `onreadystate` for a script tags.

The locking utility can also be used to create "run once" functions such as an `init()` for a class outside of its constructor. The class can simply request the lock, and on completion, refuse to release it. While this takes up a small bit of memory in the lock utility, it also guarantees only one `init()` reaches the critical section for the life of the page.

## Limitations
Building such a locking system in JavaScript is not without drawbacks. The most notable of these is the lack of support for browsers which do not support `Array.push()` (hello, Internet Explorer on a Mac). If you have this esoteric corner case, I'm stumped; there's not much you can do.

It's also unknown just how atomic the methods are in non-browsers. While brief tests in Rhino imply that `Array.push()` is properly atomic, there's no guarantee that it is in any specification.

While I couldn't find the test page, I'm happy to report that this bug no longer exists in Firefox and IE on the Mac is also no longer a concern for JS engineers.

Not Everything Adheres to JavaScript's Semi-Single-Threaded Model

var foo_count = 0;
var bar_count = 0;
var foo = function () {
  ++foo_count;
  window.setTimeout(bar, 5);
};
var bar = function () {
  ++bar_count;
  alert("foo=" + foo_count + ", bar=" + bar_count);
};

// triggers
window.setTimeout(foo, 10);
window.setTimeout(bar, 20);
// end triggers


sub process
    lock = get lock
    if not lock owner
        retry sub process in N ms (sleep)
        return

    lock was obtained
    perform critical code
    release lock

    non critical segment here
end process


JavaScript Threading

```yaml
---
slug: just-in-time-javascript
published: 2007-11-27T00:00:00-08:00
description: Chained CSS and JavaScript loading
changelog:
  - on: 2020-02-23
    note: This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal). As a result, many links in this piece were left pointing to archive.org destinations.
---
```

Web apps are heavy, and wait times suck. For every interaction you build, there's that much more code being added to the project. Eventually, it's going to be too much for the user to handle, especially if they are on dialup or slow DSL (though I suppose even an iPhone on EDGE would qualify). For example, the Cash Shop we just launched at Gaia weighed in at over 400k (~100k gzip compressed and minified). While we're a graphic heavy site, waiting for an additional 100k was needlessly painful. To boot, _there was no promise the user was even going to use that feature_, making loading all that JavaScript and CSS a really silly exercise. We needed a way to load the JS and CSS on the fly across all browsers.

Getting the JIT Loader

- [Get the most recent JIT Loader](https://gist.github.com/Jakobo/7a5aec5fcebc8f4886193b114c9ec4f0) (23k)
- [Get the most recent JIT Loader - min + packed](https://gist.github.com/Jakobo/7a5aec5fcebc8f4886193b114c9ec4f0) (5.6k)

The locking utility is also included, but if Lock exists, it won't be used. Enough thanks can't go to Ryan Grove for the [original LazyLoad](https://web.archive.org/web/20071215020306/http://wonko.com/article/527) utility. The JIT loader, like LazyLoad is released under the New BSD License.

## Putting off Loading the CSS and JavaScript
The method of loading we are specifically looking at is known as "event based loading" or "lazy loading". While the [wikipedia article on lazy loading](https://web.archive.org/web/20071215020306/http://en.wikipedia.org/wiki/Lazy_loading) is not very detailed, the summary of the article is that lazy loading allows us to wait on loading stuff until we need it. In computer science, usually the discussion revolves around expensive objects or data calls. In our case, it's all about the browser load time. There's little reason to put JavaScript or CSS into the head of a document unless we know it is 100% needed for the page. We want to put off the expensive http connection and download as long as possible.

A very practical example (and one already out there) is to defer the loading of images until they are within the viewport. Both the [jQuery Lazy Load Plugin](https://web.archive.org/web/20071215020306/http://www.appelsiini.net/projects/lazyload) and the [YUI Image Loader](https://web.archive.org/web/20071215020306/http://developer.yahoo.com/yui/imageloader/) do exactly this, reducing load times on pages by holding out on pulling down images until you need them. For these examples, scrolling the page (and moving a placeholder into the viewport) is an event that triggers the load. If this has proven successful for images and content, there's no reason this can't be applied to scripts and CSS as well.

## Loading JavaScript In Existing Frameworks
To a certain extent, Dojo and YUI have both built in a loading system for including packages before executing script. They are the [`dojo.require()` directive](https://web.archive.org/web/20071215020306/http://dojotoolkit.org/book/dojo-book-0-9/part-3-programmatic-dijit-and-dojo/functions-used-everywhere/dojo-require) and the [YUI Loader Utility](https://web.archive.org/web/20071215020306/http://developer.yahoo.com/yui/yuiloader/) respectively. These two were specifically called out because their methods of including requested files are so different.

In the case of Dojo, an XMLHttpRequest (XHR) is made to load the JavaScript package, and then is ran through `eval()` to convert the request into functional code. This lets Dojo use a package style syntax similar to Java for including libraries to run. Each call to `require()` is synchronous and blocks, meaning you can trust a directive to fully complete before getting into your code. This method is highly effective for lazily loading a package since Dojo is clever enough to not include something twice. The crutch though is the XMLHttpRequest, which is subject to any cross site scripting issues. (Note: AOL has managed to do a custom Dojo build with XHR support across domains and the process is pretty well documented, but it is not the default behavior.)

On the other end of the spectrum, the folks at Yahoo! have taken a different approach, writing to the DOM directly using the `<script>` tag. This eliminates the cross browser issues that Dojo grapples with, but trades it for the limitations of inline script writing exposing the DOM to Internet Explorer in a state when it's not "complete" in IE's mind, resulting in the dreaded Operation Aborted error. In order to use the loader, there's also a lot that needs to be done if custom modules are going to be used. (As of this writing, the YUI Loader constructor does not take the config object, so it's hard to dynamically add to the object without using `YAHOO.lang.augment()`.)

Ryan Grove built a library independent utility for including scripts, similar to YUI, called Lazy Load. It was a good strong start, and needed a steroid boost to provide the CSS support and streamlined DOM writing. Because the API was radically changed, the name was changed as well.

## Just In Time Loader (JIT Loader) Basics
In most situations, the behavior we are looking for is "on event X, load this, then load this, then load this, then execute Function X". To provide access to this streamlined interface, `JIT.startChain()` was created. Calling `startChain()` returns a method-chaining object that you can use to build up a series of JavaScript and CSS includes. The following methods are available on the object returned from the `startChain()` call.

- `addCss(urls, verifier, ie_version)`: adds a CSS File (or array of CSS files) to the execution stack. It will continue when `verifier` returns true. By default, all verifiers simply make sure the nodes have been written to the DOM, not that they have been processed internally. In the case of CSS, you can specify a specific IE version you want to apply this CSS to in order to keep your CSS files clean.
- `load(urls, verifier)`: adds a JavaScript file to the execution stack
- `loadOnce(urls, verifier)`: adds a JavaScript file to the execution stack. When fired, if the URL has already been added by another JIT load or an external source, it won't be loaded again.
- `onComplete(fn, obj, scope)`: Starts the execution chain. When the execution chain has completed, `fn` will be ran. If `obj` is suppled, it will be passed as an argument. If `scope` is `true`, `fn` will instead be directly ran in the context of `obj`.

The most primitive example of chaining would then look something like:

```js
JIT.startChain()
  .loadOnce(jsfile, function () {
    /* verifier */
  })
  .addCSS(cssfile)
  .onComplete(function () {
    // run this when everything is loaded
  });
```

If chaining is not your thing, you can also make direct calls to the JIT object (see examples at the end). After the verifier, each of the above calls takes a callback function, object, and scope; the `onComplete()` doesn't exist in the root JIT object and is used solely for the purposes of chaining. These are left available in case more advanced functionality is required.

## Optimizations
Because JavaScript is (mostly) single threaded, if there are multiple calls to the Lazy Loader, we shouldn't wait on the current code execution block to write to the DOM and clean up when it is finished. The original Lazy Load used a queue system that let multiple calls stack as pending, even if they were for different load requests. The smallest synchronous operation is the DOM write, which can be pushed out to a method and can be locked using a [spinlock](./javascript-threading) to emulate fake threads. Any events that get queued up between DOM writes can be added to the stack, and can then execute while slow scripts take a while. This speedup is surprisingly most noticeable on Internet Explorer, where events are allowed to interrupt the current JavaScript thread, to write more concurrent nodes into the DOM.

## Limitations
The verifier function that is passed in is the linchpin of the event based loading. When the verifier resolves to `true`, the loading process continues with the assigned callback. When working with CSS, this is exceptionally difficult. There are two options when dealing with event loading the CSS files, either find a testable property in the CSS, or pass in `null` for the verifier slot, and load the CSS first. While the first offers more reliability, the second tends to be more practical, since the CSS won't usually apply to anything other than nodes created by the JavaScript. A simple example of a CSS verifier would be (assuming your CSS sets the body's background image to the below mentioned URL:

```js
var verifier = function () {
  return (document &&
  document.body &&
  document.body.backgroundImage = "http://example.com/load_complete.gif")
    ? true
    : false;
};
```

The second major limitation of the system is in `loadOnce()` which requires script nodes to exist in the DOM to test if they have been properly loaded. This testing method makes it incompatible with the `dojo.require()` directive which uses the XHR to perform script includes. In the future, the verifier check might be ran once to preemptively test for component loading, but that will cause issues with testing in the `YAHOO` namespace.

## Enough Already, Examples Please!
You can see JIT in use on the website [Gaia Online for the Cash Shop](https://web.archive.org/web/20071215020306/http://www.gaiaonline.com) (under the market submenu) and eventually the same technique will be in place for the world map and other header-based events.

JIT.startChain()
  .loadOnce(jsfile, function () {
    /* verifier */
  })
  .addCSS(cssfile)
  .onComplete(function () {
    // run this when everything is loaded
  });


var verifier = function () {
  return (document &&
  document.body &&
  document.body.backgroundImage = "http://example.com/load_complete.gif")
    ? true
    : false;
};


Just In Time JavaScript

```yaml
---
slug: transactions-part-2
published: 2006-10-02
description: More on Gaia Online's "Bank Transaction Manager"
changelog:
  - on: 2020-02-23
    note: This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal). As a result, many links in this piece were left pointing to archive.org destinations.
---
```

It's been a while since I've touched on the topic, mostly because I've been busy with work for Gaia. However, this incubation period has let the transaction system mature, and it continues to hold out well with over 29 million completed transactions in the last 7 months. Since its inception, it's been adjusted to support item granting, and is currently being extended to support transactions in the Gaia Marketplace / Auction House. In the first article, I covered the surface of the transaction manager, and some of Fowler's theories which were behind it. In order to avoid recoding large parts of the site, a lock-first mentality was chosen for how to handle keeping an account up to date. This time, we're going to focus on all the players / classes of the transaction manager and how they work together.

* [Part 1: Theory](/thunked/transactions-part-1)
* **Part 2, Transaction Framework**

As in most programming languages, PHP provides support for both public and private methods within a class. In development, one of the primary objectives was to minimize the external facing interface for the end-developer, while still keeping the backend components easy for a developer to maintain. Each component was given specific tasks, responsibilities, and a list of classes that it would couple with in order to provide functionality.

## Bank Transaction Manager
The first component is the first of three end-developer facing classes. The Bank Transaction Manager is responsible for creating Transaction Entries, adding Transaction Entries to its internal stack, and then processing the entry stack while reporting on the success or failure of the process. When a Bank Transaction Manager is initialized, it creates an Account Manager object for tracking the accounts which it needs to load. As entries are created with the `createEntry()` method, they are fed to the end developer and then added back in to the system via `addEntry()`.

A Transaction Entry is nothing more than a shell object, providing a clean interface for setting an object's ID, it's attribute, quantity, and other associated properties. When `addentry()` is called, the Bank Transaction Manager uses these properties in order to decide what Account to load (via the Account Manager). This pairing is then set aside until execution.

During the execution phase, the Bank Transaction Manager tells the Account Manager to initialize all relevant accounts. It then loops through every entry/account pair and passes the entry into the account. In processing terms, these are called posting rules. Every posting rule has a boolean success, and the sum of all functions returning an ultimate success or failure for `execute()`.

If any errors are encountered, the Bank Transaction Error (a simple static class) is written to with information about the failure.

## Account Manager
In a horizontally scaled environment, a user's inventory or a row of trading information might reside on completely separate machines. To encapsulate the loading and saving of these various things is the reason for the Account Manager object. Requested accounts are queued up, and then loaded via a single call to `loadAccounts()`. Beyond simply loading accounts, the account manager should be able to

- handle information if the account has changed
- prevent data overlap by avoiding querying for duplicate data
- provide an interface for the instantiation of new Account objects

During the execution of the Bank Transaction Manager, the Account Manager is called (specifically when `BankTransactionManager::addEntry()` is used). Accounts are queued for loading as entries are added, and are then loaded at once. There are two ways to load the account data, one which uses blocking, and one which does not. Even at 4.5 million registered members, the blocking method has not been an issue, and so that is the method we'll be talking about. In an InnoDB database, a row can be locked during the select phase as part of a transaction by using the following syntax: `SELECT 'columns' FROM 'table' WHERE 'criteria' FOR UPDATE;`

The `FOR UPDATE` places a lock on the row, leaving your PHP script as the only script that has access until the lock expires. This feature is only available on InnoDB however, as the MyISAM engine supports neither row locking nor transactions.

When asked to save, the `AccountManager` calls the `isChanged()` method on every account in order to verify if it has been altered. Accounts which have been altered are then saved back to the database and the locks released.

## Transaction Entry
There's not much to say about the `TransactionEntry` object. As a shell, all it contains is a data payload. It's information consists of: "how many", "of what", "with what serial", and "adding or subtracting". To help simplify the development end, the Entry object is extended to support Game items, Regular Items, Gold, and Gift Credits. During execution, these small instruction sets are passed to the Account object.

## Account
Defined as the authoritative source regarding all data coming in and going out, these objects are modeled in spirit after the VO J2EE pattern. They exist to hold an account's data, and are decorated with methods to support the changing of that data. When the Account Manager calls `loadAccounts()`, every queued account object is asked for its database information and required fields. The sum of this data is aggregated, and passed into the `importData($fieldName, $data)` function. When the Bank Transaction Manager executes, it then takes every Transaction Entry object, and passes it directly into `postEntry($t)`. It is at this point that the heavy lifting is actually done.

Every account maintains a ruleset for how its data is to be altered, known as "posting rules". These rules are very strict, and breaking them should cause the entire transaction to fail for the user and return an error. The most common example of failure is if the posting rule contains an item type that the account does not support, such as posting gold information to an inventory account. With a single public interface, the logic of altering the account information is also internalized, protecting it from the outside world. Upon altering the account, `postEntry()` will return either `TRUE` or `FALSE`, which will be evaluated by the Bank Transaction Manager object.

Adding new accounts or extending existing accounts to add functionality is very easy. The methods exposed publicly are small, specifically `getDaoName()` and `getDaoFields()` for Gaia's database support, `importData()` for loading, `exportAccount()` for saving, `isChanged()` for the Account Manager's save required check, and `postEntry()` for altering the account's data.

## Code in Practice (End Developer Side)

```php
$btm = new BankTransactionManager($transaction);

// create entry (Account, Entry Type)
$e = $btm->createEntry("Gold.Gold");
$e->setUserId($user_id);
$e->addGold($gold_amount);
$btm->addEntry($e);

$e = $btm->createEntry("ServerGold.Gold");
$e->setUserId(ENTITY_SERVER);
$e->subtractGold($gold_amount);
$btm->addEntry($e);

try {
    $btm->execute();
}
catch (Exception $e) {
    throw $e;
}
```

More on Gaia Online's "Bank Transaction Manager"

$btm = new BankTransactionManager($transaction);

// create entry (Account, Entry Type)
$e = $btm->createEntry("Gold.Gold");
$e->setUserId($user_id);
$e->addGold($gold_amount);
$btm->addEntry($e);

$e = $btm->createEntry("ServerGold.Gold");
$e->setUserId(ENTITY_SERVER);
$e->subtractGold($gold_amount);
$btm->addEntry($e);

try {
    $btm->execute();
}
catch (Exception $e) {
    throw $e;
}


Transacting Items on Gaia Online, Part 2

```yaml
---
slug: stylizing-form-submits
published: 2006-10-28T00:00:00-08:00
description: Camino and Safari & styling type=submit
changelog:
  - on: 2020-02-23
    note: This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal). As a result, many links in this piece were left pointing to archive.org destinations.
---
```

"The buttons aren't styling" I remarked as I stared at the page in Camino and Safari. Testing on the Mac platform has been the absolute bane of my existence as of late, mostly because for some reason, these browsers in particular are very protective of their form widgets. Coming back to [Cameron Adams' Entry on Styling Form Widgets](https://web.archive.org/web/20061218112920/http://themaninblue.com/writing/perspective/2004/04/28/), I found that none of the Macromedia examples were actually working for Camino/Safari, although they looked beautiful in Firefox. The submit button simply would not take the background settings. Interesting enough though, something of `input type="button"` styled perfectly.

However, the value attribute of the button wasn't getting passed in. For the legacy code around Gaia, we have forms with more than one possible "submit" for an action. The most common case of this is preview and submit buttons we traditionally use in the forums. This left us with two scenarios:

```html
<form id="myForm" action="some.php" method="POST">
  <input type="text" name="foo" value="" />
  <input id="submitBar" type="submit" name="action" value="bar" />
  <input id="submitBaz" type="submit" name="action" value="baz" />
</form>
```

or Button tags that wouldn't pass in their value to the PHP script (and shouldn't technically perform a submit either)

```html
<form id="myForm" action="some.php" method="POST">
  <input type="text" name="foo" value="" />
  <input id="submitBar" type="button" name="action" value="bar" />
  <input id="submitBaz" type="button" name="action" value="baz" />
</form>
```

The final solution ended up being something right down the middle and using JavaScript to make up for Safari / Camino's shortcomings. Starting with a default submit button, we can replace it with a input `type="button"` node, and add a bit of JavaScript to the `onClick` to make up for the stylized button's shortcomings. The first block of JavaScript goes into the head, and the second block can either go into an `onLoad` handler or at the end of the form tag, depending on how soon you need the replacement.

```js
////////
// SwapSubmit - A Tool for Stylizing Submit Buttons on All Browsers
// Licensed under CC-GPL-2.0
// http://creativecommons.org/licenses/GPL/2.0/
////////
var RJH_SwapSubmit = {
  replacedEles: new Object(),
  replace: function (formid, eleid) {
    if (typeof eleid == "string") var ele = document.getElementById(eleid);
    else var ele = eleid;

    if (typeof formid == "string") var f = document.getElementById(formid);
    else var f = formid;

    if (ele) {
      var o = this.build(formid, ele);
      var name = ele.getAttribute("name");
      var value = ele.getAttribute("value");

      var pn = ele.parentNode;
      pn.removeChild(ele);
      pn.appendChild(o);

      if (!document.getElementById(name))
        f.appendChild(this.buildInputField(formid, name));

      this.replacedEles[eleid] = new Object();
      this.replacedEles[eleid]["name"] = name;
      this.replacedEles[eleid]["value"] = value;
      this.replacedEles[eleid]["form"] = formid;
    }
  },
  build: function (formid, ele) {
    var o = document.createElement("button");
    var f = document.getElementById(formid);

    var oldOnclick = ele.onclick;
    o.id = ele.id;
    o.className = ele.className;
    o.onclick = function () {
      if (oldOnclick) {
        if (oldOnclick() == false) return false;
      }
      RJH_SwapSubmit.submitForm(this);
    };

    return o;
  },
  buildInputField: function (formid, name) {
    var o = document.createElement("input");

    o.setAttribute("type", "hidden");
    o.id = name;
    o.setAttribute("name", name);

    return o;
  },

  // ORIGIN: onClick of a button
  submitForm: function (ele) {
    var clickId = ele.id;
    var name = RJH_SwapSubmit.replacedEles[clickId]["name"];
    var value = RJH_SwapSubmit.replacedEles[clickId]["value"];
    var formid = RJH_SwapSubmit.replacedEles[clickId]["form"];
    document.getElementById(name).value = value;
    document.getElementById(formid).submit();
  },
};
```

```js
RJH_SwapSubmit.replace("FORMID", "SUBMITBBUTTONID");
```

Replace `FORMID` and `SUBMITBUTTONID` with the relevant values.

## Drawbacks
The largest drawback hands down is the addition of JavaScript. It's not something to depend on, and many steps were taken to ensure that the enhancement is progressive in nature.

Styled images will have to be styled to reflect the change from an input to a button node, and can be referenced via `button#idNameofElement` to apply all the necessary stylization.

////////
// SwapSubmit - A Tool for Stylizing Submit Buttons on All Browsers
// Licensed under CC-GPL-2.0
// http://creativecommons.org/licenses/GPL/2.0/
////////
var RJH_SwapSubmit = {
  replacedEles: new Object(),
  replace: function (formid, eleid) {
    if (typeof eleid == "string") var ele = document.getElementById(eleid);
    else var ele = eleid;

    if (typeof formid == "string") var f = document.getElementById(formid);
    else var f = formid;

    if (ele) {
      var o = this.build(formid, ele);
      var name = ele.getAttribute("name");
      var value = ele.getAttribute("value");

      var pn = ele.parentNode;
      pn.removeChild(ele);
      pn.appendChild(o);

      if (!document.getElementById(name))
        f.appendChild(this.buildInputField(formid, name));

      this.replacedEles[eleid] = new Object();
      this.replacedEles[eleid]["name"] = name;
      this.replacedEles[eleid]["value"] = value;
      this.replacedEles[eleid]["form"] = formid;
    }
  },
  build: function (formid, ele) {
    var o = document.createElement("button");
    var f = document.getElementById(formid);

    var oldOnclick = ele.onclick;
    o.id = ele.id;
    o.className = ele.className;
    o.onclick = function () {
      if (oldOnclick) {
        if (oldOnclick() == false) return false;
      }
      RJH_SwapSubmit.submitForm(this);
    };

    return o;
  },
  buildInputField: function (formid, name) {
    var o = document.createElement("input");

    o.setAttribute("type", "hidden");
    o.id = name;
    o.setAttribute("name", name);

    return o;
  },

  // ORIGIN: onClick of a button
  submitForm: function (ele) {
    var clickId = ele.id;
    var name = RJH_SwapSubmit.replacedEles[clickId]["name"];
    var value = RJH_SwapSubmit.replacedEles[clickId]["value"];
    var formid = RJH_SwapSubmit.replacedEles[clickId]["form"];
    document.getElementById(name).value = value;
    document.getElementById(formid).submit();
  },
};


RJH_SwapSubmit.replace("FORMID", "SUBMITBBUTTONID");


Stylizing Form Submits

```yaml
---
slug: sslac
published: 2011-05-03T00:00:00.000Z
description: A JavaScript class library that lets you runtime patch your methods
changelog:
  - on: 2020-02-23
    note: This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal). As a result, many links in this piece were left pointing to archive.org destinations.
  - on: 2020-02-24
    note: You have ES6 Classes. You should certainly be using those now! `Sslac` is also long since retired, though it was in use at LinkedIn for several years because of the need to test fixes and patches for CDN-delivered code on third party sites via the browser console. Ultimately, this was a lot safer than giving the framework a "switch" flag that could retrieve runtime JS from an arbitrary URL.
---
```

The world of JavaScript is rich with ways to bring typical OOP constructs to a highly prototyped language. The world probably doesn't need another library either. Further, it probably doesn't need an entire blog post talking about its features and how it can be used. Nevertheless, the hat is being tossed into the ring. Welcome Sslac, a (somewhat) backward approach to class instantiation in JavaScript.

If you're not keen on the intro documentation, you can [download Sslac from github](https://github.com/jakobo/sslac), view the source, etc. The Short List (features)

- Natural chaining syntax to define an object
- Compatible with `instanceof` operators
- Support for superclass methods
- Change definitions of objects after creation
- Support for namespacing and `noConflict`
- Common JS 1.1 Modules compliant
- Multiple class types (static, instance, and functions)

## A Dual Object Structure

The most important thing to remember in Sslac is that when working from the Sslac chain, you're working on what's internally called the "ObjectRef". This reference isn't the object your code will instantiate, but rather a souped up dictionary with some helpful setters and getters. This syntax allows you to get a method that you've already defined somewhere along the extension chain, and then swap it for something new or patch new code over the old. We'll actually get to that in a bit, but first, the class basics.

For people who have worked with JS class libraries, the syntax should feel familiar. With lowercase versions of the words reserved (static and extends come to mind), Uppercase was used instead. An unexpected benefit of this seems to be that code written is a bit easier to read. For those who have played with Prototype or [Resig's Simple JS Inheritance](https://web.archive.org/web/20110507092802/http://ejohn.org/blog/simple-javascript-inheritance/), the syntax shouldn't feel too unwieldy either.

## Runtime Mixins and Monkeypatching

I hear it's also called "duck punching," which sounds way more cool. One thing Sslac really excels at is the ability to alter code at runtime without modifying the original source code. Much like Prototype's addMethod functionality, these runtime methods cascade into inherited objects. Unlike the other frameworks, however, Sslac provides a set of helper methods to get at the original definitions. To change a method's definition at runtime, you can retrieve the original object's definition, and then swap out a method.

    (this code sample went missing in the many migrations)

In the above, we've enabled the Person object to flip out, and enabled the Ninja object to kill people. Thankfully, only Ninjas (which extend Persons) will be able to [flip out and kill people](https://web.archive.org/web/20110507092802/http://www.realultimatepower.net/index4.htm), but that's the purpose of ninjas anyway. Prototype inheritance means that, by adding the `flipOut` method to Person, Ninja will automatically get it. The Person object's probably feeling left out though. And if it's someone else's Person object, you may not want to re-implement their method entirely. This is where that whole "duck punching" concept comes in.

Using `definitionOf` and `getMethod`, we can retrieve the original implemented method and call it using the old school `apply` method, passing the current object along with the arguments array.

## In the Real World

Perhaps the most compelling real-world use case for Sslac comes from development at LinkedIn. With our current framework deployed on high profile sites, we can take a bug we want to fix, write the patch, and then use a tool like Greasemonkey to add our own custom code on top. Bugs can be verified, partner implementations can be tested to ensure there are no regressions, all without either us or the end user developers having to change production code. This provides an invaluable mechanism, allowing code changes to be verified out in the wild, on live sites, without impacting any end user experience. If nothing else, the monkeypatching alone has made the 2k library (minified, pre gzip) well worth its cost.

[Sslac on Github](https://github.com/jakobo/sslac)


You have ES6 Classes. You should certainly be using those now! `Sslac` is also long since retired, though it was in use at LinkedIn for several years because of the need to test fixes and patches for CDN-delivered code on third party sites via the browser console. Ultimately, this was a lot safer than giving the framework a "switch" flag that could retrieve runtime JS from an arbitrary URL.

A JavaScript class library that lets you runtime patch your methods

Sslac: A Runtime Patchable Class Library

```yaml
---
slug: data-liberation-for-linkedin
published: 2015-03-15T00:00:00.000Z
description: Free your data
changelog:
  - on: 2020-02-23
    note: This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal). As a result, many links in this piece were left pointing to archive.org destinations.
---
```

Hackdays at LinkedIn are special. People get to try new ideas and technologies. I personally enjoy seeing if I can push people's buttons and make them think, so I am a sucker for the "most controversial" category we always do. In 2012, [UnLinked](http://jakobo.github.com/UnLinked/) ([github](http://jakobo.github.com/UnLinked/)) was lucky enough win one of the categories. It may end up on LinkedIn Labs eventually, but that's not really the point of this post. This post is all about how easy it is to export your LinkedIn data using the JavaScript APIs as part of the LinkedIn Platform. If you want to run your own local copy of UnLinked, just fork the github project above. Special thanks to my collaborator on this Hackday project [Efrat Orkin](http://www.coroflot.com/efratorkin) who's design ability put incredible personality and purpose into the project . This is the Hackday project, broken down from idea to its final forms.

![An image of the UnLinked project, showing an obvious like to download your data in an easy to use JSON format](https://user-images.githubusercontent.com/1795/146730319-a01f9b8d-7017-4d17-9589-b6e235eabf03.png)

## Free Your Data

I'm a huge fan of the [Data Liberation Front](http://www.dataliberation.org/), a group within Google that obsesses over freeing the data. If you ever want to know how to export/access your data from Google's data centers, this single site details everything. The goals are simple; you want to create

1. a way to export your data in an open, interoperable format…
2. that shouldn't cost the user anything…
3. and take as little time as possible

This is a pretty strong edict. Any company with APIs should be able to do this; LinkedIn is no exception. The [LinkedIn Developer Site](https://developer.linkedin.com/) documents the APIs we have available. During brainstorming, I wanted an application that makes it one-click-simple to get your profile information and download it. Docs in hand, it was time to create a "Sign in With LinkedIn" button and start accessing data.

## Building UnLinked

UnLinked itself consists of a single HTML page. Everything we did was built on the LinkedIn JavaScript APIs. That is: request access, make N calls to the API to get all the required data, and bundle the whole thing up into a JSON format. To help the application feel fast, we broke all the calls up into smaller chunks using LinkedIn's `API.Raw()` call. The results are then mashed together in the final payload. When we've got all the parts, we can do the download. For the GitHub version, a client side result is displayed. It's also possible to use the simple PHP script to trigger a download of the JSON data. The total engineering took about 3 hours from no API key to completion and completed debugging.

One of the more interesting things done in the application is the liberal use of the `.Raw()` API call. While there are shortcut methods to get various data components, `Raw` affords us several advantages. First, you don't have to accept the defaults. Since this application is designed to grab as much data as possible, it needed to ask for more than just the default commonly used fields. Second, as the number of URLs grows, it becomes very easy to change the data structure to allow an unlimited number of attributes to be retrieved, without adding much additional code. Finally, some endpoints just don't have convenience methods build in yet. For those items, we needed some way to get at the data- the original purpose `Raw` was built. Since our URLs don't change in the v1 API, this is a pretty stable solution that also affords us the power we're looking for.

## The Result

Counting the styling and graphics creation, Efrat and I spent about 10–12 hours total, which was less than a day's work for us combined. The JavaScript APIs for LinkedIn are very powerful, and the more we move as a Web to client side solutions, the more likely JavaScript APIs will move from away from their Hackday status and become part of client side models in MVC architectures.


Data Liberation for LinkedIn

```yaml
---
# This frontmatter will be used by codedrift
slug: useful-git
published: 2015-03-18T00:00:00.000Z
description: There are some good git commands, and there are great git commands
changelog:
  - on: 2020-02-23
    note: This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal). As a result, many links in this piece were left pointing to archive.org destinations.
---
```

I love git. It and its other distributed version control systems have changed the way we think and write about code. Not too long ago, we used to commit giant monoliths of code to SVN locations. Complex environments often had to dedicate at least one person to managing branches. It was the complete opposite of good. I've been using git now for over 8 years, and I'm still learning tips and tricks that make my day to day development easier and better. These are some of my favorite commands.

## git bisect: When did the code go wrong?

Eventually something will break, and if you're unlucky, it will only happen after a long stream of commits. The git bisect command allows you to begin searching for that commit where things went south. The command looks like:

    git bisect start
    git bisect bad HEAD
    git bisect good aecdb123
    # and so on...

You'll need to specify a "bad" revision (often `HEAD`), a "good" revision (perhaps from a tag or git log), and then you're off to the races. Git will select a commit exactly halfway between "good" and "bad". So you'll test your code. If it worked, type `git bisect good`, and if it didn't type `git bisect bad`.

Every time you tell git if things are good/bad, it will cut the remaining revisions down by half. Before you know it, you'll have zeroed in on the commit that is causing everything to fail. Check the commit log, see what happened, and you'll usually be 99% of the way towards fixing your broken build. All of the bells and whistles that make bisect even cooler are available on the [git-bisect docs page](http://www.kernel.org/pub/software/scm/git/docs/git-bisect.html).

## pull + rebase: 2 for 1

You can pull. You can rebase. Or you can do both in a single line

    git pull --rebase origin main

## git config: Type less with aliases

Maybe you don't like reaching up for the hyphen key `-` and want a simple command to pull and rebase at the same time. `git config` lets you set any of the config options. There are a lot, but for now we'll just show off an alias.

    git config --global alias.pr 'pull --rebase'

From this point forward, you can type `git pr` to pull & rebase. Your hyphen key will thank you!

## git-reflog: Undo nearly anything

`git reset` (with the hard flag) is probably one of the scariest operations you can do. Just like that, it could appear your entire repository was catapulted back in time. This can be pretty great… as long as you didn't need any of those changes. It only took me accidentally blowing away my changes once before I did `git reset` in new branches only. This was before I learned about the "reflog".

The `git reflog` command spits out a sequence of operations.

    1907832 HEAD@{0}: checkout: moving from master to sample
    1907832 HEAD@{1}: commit: working on git article
    809189a HEAD@{2}: commit: final title change
    1887f64 HEAD@{3}: commit: minor amendment to iframe article
    6c8e64c HEAD@{4}: commit: launch iframe blog, added to git blog
    b47369b HEAD@{5}: pull origin master: Fast-forward
    7f66541 HEAD@{6}: commit: changed to hashes for markdown, new post
    b9e19d0 HEAD@{7}: commit: converting to pygment happy land
    432a308 HEAD@{8}: rebase finished: returning to refs/heads/master
    432a308 HEAD@{9}: rebase: more drafting of the creed post

It turns out that it's damn-near impossible to lose something in git. As you commit, rewind, replay, and revert, git keeps a record of every single thing you've done. Now, let's reset to some arbitrary distance ago, destroying all my hard work.

    git reset --hard b9e19d0
    # HEAD is now at b9e19d0 converting to pygment happy land

Sure, I'm back in "pygment happy land", but I just lost 7 commits worth of work. `git reflog` knows you better than you think; it has your history of changes, including your foolish reset:

    b9e19d0 HEAD@{0}: reset: moving to b9e19d0
    1907832 HEAD@{1}: checkout: moving from master to sample
    1907832 HEAD@{2}: commit: working on git article

A quick reset back to `1907832`, and everything is back as if **nothing was ever destroyed**. There's a line added to your reflog for the reset you made too! This makes `git-reflog` incredibly powerful. It's also comforting to sometimes just look at the reflog and know git's got your back.

---

Most of these tips were pulled from notes I took while learning git. I then learned there is an entire website dedicated to improving your git-fu. [http://gitready.com/](http://gitready.com/) has examples, tricks, tips, and even more amazing things that go beyond this.


There are some good git commands, and there are great git commands

Useful git

```yaml
---
# This frontmatter will be used by CodeDrift
slug: on-take-home-coding
published: 2015-03-19T00:00:00.000Z
description: I'm Sorry, I Won't do Your Take Home Coding Exercise
changelog:
  - on: 2020-02-23
    note: This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal). As a result, many links in this piece were left pointing to archive.org destinations.
  - on: 2020-03-15
    note: I'm more conflicted on this original piece in light of the pandemic. Interviewing against a dry erase board was already a bad measure; adding a remote code session just increases the number of things that can go wrong exponentially. That said, I think giving someone an activity they can perform in an _hour_ and having the candidate share their screen is a suitable alternative. If you absolutely must give people take home exercises, for whatever reason, consider compensating them for their time.
---
```

The instant we got rid of our "take home exercise" at LinkedIn, we saw an increase in the number of qualified candidates engaging in our interview process. We seem to be going against the grain here, as many large companies seem to want a "take home coding exercise" as part of the process before candidates even move into a more formal interview. I'm here to hopefully stop you, the reader, from falling into this trap.

## Noise, Not Signal

I totally understand the company side of this; take home activities offer a really effective way to get the candidate producing code which can be screened at the luxury of the company. You can save your already hired employee sometime by only interviewing people who have produced code that meets a minimum bar. You can even use this code during the on-site interview for follow up questions and diving deeper into the work in order to understand the candidate's thought process. By raising the barrier to entry, you've also ensured only people who really want to work at your company will proceed with the interview process.

What nobody counted on was the Internet. For all the time spent on building a question, one quick post to a site like Glassdoor can remove all of the value the company hoped to gain from a take home exercise. Building solid, reliable, and consistently measurable take home activities for candidates takes a very large and constant investment of time. And all of this before a single exercise has actually come back to your company.

If the interviewer thinks the code is flawless, then the question is probably on the Internet in some form. If the code was absolutely terrible, chances are that the recruiter wouldn't have provided the take home activity in the first place. This leaves everything in between, where an interview is still required in order to assess capability.

## The Candidate's Priorities

Things are even worse when one considers the other side of the interviewing relationship. Sending a take home activity to an engineer sends a very clear message that your company doesn't value their time. "Please build for me this fictitious thing", you say. "Complete this task, and we may let you talk to an engineer." That's a large hurdle to someone who doesn't even know if they would really like to be a part of your company in the first place.

Passive candidates don't have time, as they were quite happy with what they are doing currently(hence passive). These candidates often are spending their non-work time having families, pursuing hobbies, or contributing to open source projects. For these people, nothing is a bigger buzzkill than an activity that offers nothing to the candidate other than more of the same work they do.

Active candidates have the time. Since they are actively looking for a new opportunity, it's safe to say there's definitely more than one company they are talking to. In this scenario, asking the candidate to take 3 days for a coding activity is three days everyone else is actually talking to the candidate, assessing fit, and selling the candidate on the awesomeness of their respective companies. In these scenarios where an active candidate hedges their bets, take home activities rarely come back.

There are thousands of opportunities in our industry. When a company sends the "we don't trust you, so prove yourself" email, that company gets mentally shuffled lower in the queue for both active and passive candidates alike.

## The Work is Out There

It doesn't have to be like this. Our industry has already built LinkedIn, Stack Overflow, GitHub, BitBucket, TopCoder, and hundreds of other places where people can show their work. As a recruiter or engineer, ask them for code they would like to share. If there is no public samples available or offered, then it's worth sitting the candidate down with an engineer to perform a small directed coding task. Not only will the company get a chance to see the candidate's development process, the candidate will have the opportunity to learn more about what it is like to work with the team.

As for me, I will likely pass on your take home coding exercise. I hope my work can stand on its own.

I'm more conflicted on this original piece in light of the pandemic. Interviewing against a dry erase board was already a bad measure; adding a remote code session just increases the number of things that can go wrong exponentially. That said, I think giving someone an activity they can perform in an _hour_ and having the candidate share their screen is a suitable alternative. If you absolutely must give people take home exercises, for whatever reason, consider compensating them for their time.

I'm Sorry, I Won't do Your Take Home Coding Exercise

On Take-home Coding

```yaml
---
slug: phpbb-permissioning
published: 2004-12-02T00:00:00-08:00
description: How Gaia's volunteer moderator system outstripped phpbb's capabilities
changelog:
  - on: 2020-02-23
    note: This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20041207010434/http://felocity.org/) site. Writing on the internet was pretty different back then. As an aside, this was how I ended up working for one of the coolest social gaming companies at the time.
---
```

Currently, I've been doing a lot of work with the way phpbb handles its permissions. The method is a group based Moderator scheme, which then uses a different user attribute for most moderation functions. Many people know about Gaia's two moderator groups, and this is actually more in line with how phpbb intended permissions to be. Give a group power to modify forum X, give another group power to modify forum Y. The problem is, all moderation functions check to see if you are an admin. In phpbb's architecture, there is no admin "group". The only reasoning I could find for it was that it simply wasn't thought of. Many other large scale systems (Xoops comes to mind) use an administrative group just like any other group. Permissions are then applied on the group level.

So then, in centralizing the permissioning model, we see two primary approaches. The first of these is to centralize permissions into groups, and then add users to groups. However, with over 1.3 million users, the group table will grow very large, very quickly. However, you could change permissions for an entire group of people by just adjusting the group. The other way that came to mind was to centralize the permissions on the users themselves. This would allow for "groups of permissions" instead of "groups of users". From a management perspective, this actually isn't the better of the two solutions. However, assuming a static package of permissions relative to a user's power on the site, it makes sense to simply give a user a level / value / package. You could still potentially build a model for a finer granularity of control, but in the case of Gaia, a moderator / admin / etc has a predefined series of tools. The likelihood of adding someone who would mod area A but not area B is actually small (compared to the total user base) and so these individual cases are realistic enough to handle in a separate table.

The other piece of rationale to a user-centric permissioning system is its need to be both forward-compatible with future development, and yet backwards compatible with everything else that has been written. This is where the nightmare truly begins. Several pieces around the site need to be reengineered to the new permission model, including the way our forum tree is built, the piece that says if you can view a topic or not, forum viewing, jumpboxes, etc. However, there are prefabricated templates and expected results. So the new code has expected input, and output, but a completely different internal engine. Essentially then, we are emulating phpbb's authorization functions while everything under the hood changes. Thankfully, all future developments already include the user-centric model, so the biggest overhaul is in the forums.

I've got some 3 pages of notes about ideas of where to change code. It's time now to see if anything out of these notes will pan out.

This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20041207010434/http://felocity.org/) site. Writing on the internet was pretty different back then. As an aside, this was how I ended up working for one of the coolest social gaming companies at the time.

How Gaia's volunteer moderator system outstripped phpbb's capabilities

phpbb Permissioning

```yaml
---
slug: most-minteresting
published: 2005-10-30T00:00:00-08:00
description: Turning Mint's traffic data into a feed of posts
changelog:
  - on: 2020-02-23
    note: This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal). As a result, many links in this piece were left pointing to archive.org destinations.
---
```

Let's get right to it- I've been blogging a lot about [Mint](http://www.haveamint.com/) lately, and there really is a good reason for it all. As a PHP/MySQL person, I love to tinker. You may have noticed a new thing where the portfolio used to be (if you're not the type for RSS). Thanks to Shaun's Mint and the good plugin folks of [Textpattern](http://www.textpattern.com/), I've written up a little something I like to refer to as the "Most Mintersting Articles". Puns and jokes aside, this is a pretty robust plugin for TXP, and includes its own caching module.

The need was a simple enough one, I wanted more exposure for the more interesting things on the site I've written. Some articles get a lot more reads than others, and some articles get a lot more comments. Since Mint is really good at collecting where on my site people went, it made sense to design a plugin to focus on that aspect. Additionally, Mint rolls its data and deletes old logs so the "freshest" pages are always going to update with time.

## Caching the Minty Joy
If I could walk out of here with a `LIKE %` in my SQL and run it every single time no matter what, I would have. However, that's a bad idea on a small machine with a big database table. With a little bit of caching, we can eliminate the server strain that would otherwise be caused by the plugin. (And if it's really extreme, removing the wild-cards from the resource variable can remove the problem altogether.) The caching concept is actually based on a very old method we used for Gaia Online, put together with a bit more modern piece from the new database code we rolled out. It is based on the idea that if a query should be cached, it's SQL should never change. This means we can base the filename for the cache off of the SQL query. Since we have a very small number (1) of queries in this case, a CRC32 will suffice.

## The Goods
The rest really just needs to be seen to be believed. Of course, you need Mint (link at the top there) to make use of this, but Shaun made some awesome software, so it's well worth the price. I'd like to think of this as another good reason to get fresh and minty!

Update: Version 0.2 now supports the Mint object if Mint is on a different database than TXP.

Download minteresting v 0.2 available as a [gist](https://gist.github.com/Jakobo/f460f8f08cf92379ca47fcea4b3220f0)

Turning Mint's traffic data into a feed of posts

Most Minteresting

```yaml
---
slug: line-numbers-in-eval
published: 2015-03-16T00:00:00.000Z
description: You should never do this. But "never" is a bit too absolute for engineering
changelog:
  - on: 2017-10-01
    note: Better alternatives than `eval` finally exist with `await import()` syntax. You should definitely explore those. Additionally, LinkedIn has sunset [Inject](https://github.com/linkedin/inject) on their production site now that many viable alternatives exist. Hoo-ray modern web development!
  - on: 2020-02-23
    note: This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal). As a result, many links in this piece were left pointing to archive.org destinations.
---
```

When it comes to executing code, `eval()` is a [terrible idea](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/eval). However, somewhere in your coding career, you will need to evaluate code.

And unfortunately, if there's an error in the code, any line numbers are going to point to the invocation of `eval()` and not the erroring line within the body itself. Lucky for us, `<script>` tags correctly report erroring lines with line numbers.

Before we go any further, this is an obligatory **don't use eval**. It should be an option of last resort. Your Content Security Policy is hopefully already preventing you from doing this.

## Trying to not Evaluate: `.innerHTML` or `.text`

The first attempt was to set `innerHTML` to your JavaScript, and then place that node onto the page via `appendChild`. If you are only concerned with modern non-Microsoft browsers, you'd be done at this point.

```js
// ⚠️ Obligatory THIS IS EVAL DO NOT DO THIS
function createScriptNode(code) {
  var scr = document.createElement("script");
  scr.type = "text/javascript";
  scr.innerHTML = code;
  return scr;
}
```

Internet Explorer won't execute the JavaScript inside of this script tag, even though the `innerHTML` property is set. However, it uniquely [supports the "text" property](http://msdn.microsoft.com/en-us/library/ie/ms535892(v=vs.85).aspx), which no other browsers seem to support. When set, scripts in IE will execute once appended to the DOM. A few changes to our above script, and we have a "safe" method. Borrowing from the idea of [feature testing for things only IE supports](./enhancing-for-bugs), we'll actually feature test against this text property, falling back to alternate versions as needed.

```js
// ⚠️ Obligatory THIS IS EVAL DO NOT DO THIS
function createScriptNode(code) {
  var scr = document.createElement("script");
  scr.type = "text/javascript";
  try {
    scr.text = code;
  } catch (e) {
    try {
      scr.innerHTML = code;
    } catch (ee) {
      return false;
    }
  }
  return scr;
}

```

## Tying it Together

Further optimizations can be used to pre-select the best insertion method. In [inject](https://github.com/linkedin/inject/blob/68c343180ed3a08dffb0ad445fe45d70908683e1/src/executor.js#L129), we wrap the code we want to execute within a function declaration and assignment. This enables us to store the results of the `eval` so that modules can then be executed on demand. The below example is just a simple JSON evaluator as a proof of concept. Like all things `eval`, you should always be cautious with invoking it on items that are not 100% in your control.

```js
// ⚠️ Obligatory THIS IS EVAL DO NOT DO THIS
var createScriptNode = (function () {
  var testScr = document.createElement("script"),
    property = "innerHTML";
  testScr.type = "text/javascript";
  try {
    testScr.text = ";";
    property = "text";
  } catch (e) {}
  return function (code) {
    var scr = document.createElement("script");
    scr.type = "text/javascript";
    scr[property] = code;
    return scr;
  };
})();

var evalJSON = function (jsonString) {
  var exec = ["window.results =", jsonString],
    node = createScriptNode(exec),
    results;
  document.body.appendChild(node);
  results = window.results;
  delete window["results"];
  return results;
};
```

While this code definitely makes it possible to do more harm than good (we're stepping around JSLint/Hint `eval` checks), the upside is huge when you're evaluating code and need to understand at what line something is failing on. In the case of a module loading system, having both the upsides of a lazy `eval` and the  embedded script tag is a huge win.


Better alternatives than `eval` finally exist with `await import()` syntax. You should definitely explore those. Additionally, LinkedIn has sunset [Inject](https://github.com/linkedin/inject) on their production site now that many viable alternatives exist. Hoo-ray modern web development!

You should never do this. But "never" is a bit too absolute for engineering

// ⚠️ Obligatory THIS IS EVAL DO NOT DO THIS
function createScriptNode(code) {
  var scr = document.createElement("script");
  scr.type = "text/javascript";
  scr.innerHTML = code;
  return scr;
}


// ⚠️ Obligatory THIS IS EVAL DO NOT DO THIS
function createScriptNode(code) {
  var scr = document.createElement("script");
  scr.type = "text/javascript";
  try {
    scr.text = code;
  } catch (e) {
    try {
      scr.innerHTML = code;
    } catch (ee) {
      return false;
    }
  }
  return scr;
}



// ⚠️ Obligatory THIS IS EVAL DO NOT DO THIS
var createScriptNode = (function () {
  var testScr = document.createElement("script"),
    property = "innerHTML";
  testScr.type = "text/javascript";
  try {
    testScr.text = ";";
    property = "text";
  } catch (e) {}
  return function (code) {
    var scr = document.createElement("script");
    scr.type = "text/javascript";
    scr[property] = code;
    return scr;
  };
})();

var evalJSON = function (jsonString) {
  var exec = ["window.results =", jsonString],
    node = createScriptNode(exec),
    results;
  document.body.appendChild(node);
  results = window.results;
  delete window["results"];
  return results;
};


Line Numbers in eval

```yaml
---
slug: sifr-and-floats
published: 2005-04-14T00:00:00-08:00
description: A flash of inline content in your (flash) replacement
changelog:
  - on: 2020-02-23
    note: This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal) site. Writing on the internet was pretty different back then.
  - on: 2020-02-23
    note: Flash isn't a technology we should use anymore. Modern web fonts have made this article completely obsolete. It's a good thing.
  - on: 2021-02-20
    note: Flash is officially dead
---
```

I ran into a snag while preparing for the May 1 reboot, involving [sIFR](https://web.archive.org/web/20050507121849/http://www.mikeindustries.com/blog/archive/2005/02/sifr-2.0-release-candidate-4). I was attempting to replace a H2 element in the following two columns. They were designed to display side by side using opposing floats. However, someone pointed out that in Firefox for Windows only (now there's a new browser oddity for you) the flash text would cling to the top of the container element, in this case, content. Here was the initial CSS being used:

```css
#content #recentEntries {
  float: left;
  display: block;
  width: 200px;
}

#content #latestProject {
  float: right;
  display: block;
  width: 200px;
}
```

The problem was solved by making use of absolute positioning within a relative container. This solved the weird flash "flicker" that was occurring using the sIFR technique. The final CSS looked like:

```css
#content #recentEntries {
  position: absolute;
  left: 0;
  display: inline;
  width: 200px;
}

#content #latestProject {
  position: absolute;
  left: 256px;
  display: inline;
  width: 200px;
}
```

I don't think I will be the only person who ever goes the route of floats and runs into this peculiar snag. Once I complete the Felocity design, I will also put up the alternate stylesheet that fully demonstrates the flicker effect. In the meantime though, it's back to packing boxes. I've been preparing for a move to the San Jose area in the interest of not driving an hour every time I need to go to San Jose—which translates to 3-5 days a week. There's a good number of boxes lying around and in my room. My original plan of a "few short trips and one big trip" is quickly becoming anything but a few. It seems like you always have more stuff when it's in boxes.

This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal) site. Writing on the internet was pretty different back then.

Flash isn't a technology we should use anymore. Modern web fonts have made this article completely obsolete. It's a good thing.

A flash of inline content in your (flash) replacement

#content #recentEntries {
  float: left;
  display: block;
  width: 200px;
}

#content #latestProject {
  float: right;
  display: block;
  width: 200px;
}


#content #recentEntries {
  position: absolute;
  left: 0;
  display: inline;
  width: 200px;
}

#content #latestProject {
  position: absolute;
  left: 256px;
  display: inline;
  width: 200px;
}


sIFR & Floats

```yaml
---
slug: ajax-prototype-scope
published: 2006-04-13T08:00:00.000Z
description: Avoiding AJAX Scope Issues in Prototype.js
changelog:
  - on: 2020-02-23
    note: This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal). As a result, many links in this piece were left pointing to archive.org destinations.
---
```

I haven't seen this covered much, but I really felt it was worth a mention. A lot of people have worked with the [prototype library](https://web.archive.org/web/20060426013445/http://prototype.conio.net/), including myself. One of the most challenging things I've had to do was create an object that could make an AJAX request, and then internalize the response. Calling a function out on the global scope is messy and in poor taste if you can avoid it. I thought back to my Java classes, and remember an almost reflexive style of programming. About an hour later (including reading through prototype's source) I had a working model. The major players in this design are the `XMLHttpCatchobject` and the `Ajax.Request` object. Our goal is to call a sending function inside of an instance of `myObj`, and then have the response be ran in the context of `myObj`. By default, you cannot refer to `this` inside of the callback, because of scope issues. The solution is to encapsulate the request, and then have a way for the response to find its way back home. We do this by passing the scope down into the `XMLHttpCatch object`. Hopefully other people will find this useful, as occasionally it is not practical to have globally scoped functions for `onComplete`, and you want access to all the proper `this.methodName()` calls. I thought about moving the catch object into `myObj` as a function definition during initialize, but the current model is a bit more reusable.

```js
Abstract.XMLHttpCatch = Class.create();
Abstract.XMLHttpCatch.prototype = {
  initialize: function (opts) {},
  receive: function (response, scope) {
    scope.receive(response);
  },
};

Abstract.myObj = Class.create();
Abstract.myObj.prototype = {
  initialize: function (opts) {
    for (var prop in opts) {
      this[prop] = opts[prop];
    }
    this.XMLHttpCatch = new Abstract.XMLHttpCatch();
  },

  receive: function (response) {
    this.response = response.responseText;
    alert("Response Text: " + this.response);
  },

  send: function () {
    // get a local name for our catch object
    XMLHttpCatch = this.XMLHttpCatch;

    // get a local name for "this" (what we want to run the response under)
    scope = this;

    // prototype does the rest
    ajax = new Ajax.Request(
      "http://www.myurl.com/path/to/my.php", // URL
      {
        // options
        method: "get",
        onComplete: function (resp) {
          XMLHttpCatch.receive(resp, scope);
        },
      }
    );
  },
};

example = new Abstract.myObj();
example.send();
```

Avoiding AJAX Scope Issues in Prototype.js

Abstract.XMLHttpCatch = Class.create();
Abstract.XMLHttpCatch.prototype = {
  initialize: function (opts) {},
  receive: function (response, scope) {
    scope.receive(response);
  },
};

Abstract.myObj = Class.create();
Abstract.myObj.prototype = {
  initialize: function (opts) {
    for (var prop in opts) {
      this[prop] = opts[prop];
    }
    this.XMLHttpCatch = new Abstract.XMLHttpCatch();
  },

  receive: function (response) {
    this.response = response.responseText;
    alert("Response Text: " + this.response);
  },

  send: function () {
    // get a local name for our catch object
    XMLHttpCatch = this.XMLHttpCatch;

    // get a local name for "this" (what we want to run the response under)
    scope = this;

    // prototype does the rest
    ajax = new Ajax.Request(
      "http://www.myurl.com/path/to/my.php", // URL
      {
        // options
        method: "get",
        onComplete: function (resp) {
          XMLHttpCatch.receive(resp, scope);
        },
      }
    );
  },
};

example = new Abstract.myObj();
example.send();


AJAX Scope in Prototype.JS

```yaml
---
slug: enhancing-for-bugs
published: 2015-03-14T00:00:00.000Z
description: Sometimes, instead of feature detection, you can get away with "bug detection"
changelog:
  - on: 2020-02-23
    note: This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal). As a result, many links in this piece were left pointing to archive.org destinations.
---
```

In the days of Internet Explorer 6, it was common to lament the problems of the browser nobody wanted to support. Some of the bugs that developers have uncovered are well documented, some of them not. This falls more into the later camp. It's fix, while trivial, serves as a great reminder about how to build software that works in every browser.

Let's get the basics out of the way first. In every browser on the planet (except IE6), it is possible to create a checkbox in IE6 that is by default selected using everyday JavaScript.

```js
var cb = document.createElement("input");
cb.type = "checkbox";
cb.checked = true;
```

A simple checkbox
The above code would render the checkbox once you append the checkbox to the DOM, but no matter how one tries, the box won't actually start in the checked state. While it's technically possible to [sniff IE6 using conditional comments](http://dean.edwards.name/weblog/2006/11/sandbox/) and react accordingly, trying to detect a browser based on its signature will always be a poor choice and a last resort.

Internet Explorer's documentation on `document.createElement` shows that before IE9, it was possible to render an element using the angle brackets, `<` and `>` respectively. When rendering out an entire tag with angle brackets, IE6 properly sets the "checked" property. Since standards compliant browsers don't allow angle brackets, we can start with a non-standard solution, falling back to the W3C method on failure.

```js
// IE6 requires a checkbox to be made differently
try {
  var cb = document.createElement('<input type="checkbox" checked>');
} catch (e) {
  var cb = document.createElement("input");
  cb.type = "checkbox";
  cb.checked = true;
}
```

By applying the philosophy of feature detection to something that is (by all accounts) a bug, we can keep true to our development principals in the browser.


Sometimes, instead of feature detection, you can get away with "bug detection"

var cb = document.createElement("input");
cb.type = "checkbox";
cb.checked = true;


// IE6 requires a checkbox to be made differently
try {
  var cb = document.createElement('<input type="checkbox" checked>');
} catch (e) {
  var cb = document.createElement("input");
  cb.type = "checkbox";
  cb.checked = true;
}


Enhancing for Bugs

```yaml
---
slug: fake-germanizer
published: 2008-11-13T00:00:00.000Z
description: Long words can make short work of your design
changelog:
  - on: 2020-02-23
    note: This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal). As a result, many links in this piece were left pointing to archive.org destinations.
---
```

I ran into a problem where I needed to guess how something would look in German, without knowing a single drop of German. So, I pulled up Mozilla's XPath documents and made a small bookmarklet. It's far from perfect, but it definitely has its uses when you're trying to estimate how items will look before they reach a translation team... even if the first use was messing with my Twitter feed.

[Fauxgermanhausen das Pagen! (bookmarklet)](javascript:%28function%28%29%7Bvar%20prefixes%3D%5B%27%27%2C%27glocken%27%2C%27das%27%2C%27borfa%27%2C%27maushe%27%2C%27uber%27%5D%2Csuffixes%3D%5B%27%27%2C%27hausen%27%2C%27%20die%20vander%27%2C%27gleuten%27%2C%27noshan%27%2C%27flagellan%27%2C%27mek%27%2C%27dak%27%2C%27en%20das%27%2C%27ga%27%5D%2CxPathResult%3Ddocument.evaluate%28%27.//text%28%29%5Bnormalize-space%28.%29%21%3D%27%27%5D%27%2Cdocument.body%2Cnull%2CXPathResult.ORDERED_NODE_SNAPSHOT_TYPE%2Cnull%29%2Ci%2CtextNode%2Ccnt%2Cout%2Cj%2Cpfx%2Csfx%3Bfor%28i%3D0%2Cl%3DxPathResult.snapshotLength%3Bi%3Cl%3Bi++%29%7BtextNode%3DxPathResult.snapshotItem%28i%29%3Bif%28textNode.parentNode.nodeName.toLowerCase%28%29%3D%3D%27script%27%7C%7CtextNode.tagName%3D%3D%27style%27%29continue%3Bcnt%3DtextNode.data.split%28/s/g%29%3Bout%3D%5B%5D%3Bfor%28j%3D0%3Bj%3Ccnt.length%3Bj++%29%7Bif%28cnt%5Bj%5D.replace%28/%5Bs%5D/g%2C%27%27%29%3D%3D%27%27%29continue%3Bpfx%3D%28%21Math.floor%28Math.random%28%29*10%29%29%3F%27%27%3Aprefixes%5BMath.floor%28Math.random%28%29*prefixes.length%29%5D%3Bsfx%3D%28%21Math.floor%28Math.random%28%29*10%29%29%3F%27%27%3Asuffixes%5BMath.floor%28Math.random%28%29*suffixes.length%29%5D%3Bout.push%28pfx+cnt%5Bj%5D+sfx%29%3B%7DtextNode.data%3D%27%20%27+out.join%28%27%20%27%29+%27%20%27%3B%7D%7D%29%28%29%3B)

```js
    (function () {
      var prefixes = ["", "glocken", "das", "borfa", "maushe", "uber"],
        suffixes = [
          "",
          "hausen",
          " die vander",
          "gleuten",
          "noshan",
          "flagellan",
          "mek",
          "dak",
          "en das",
          "ga",
        ],
        xPathResult = document.evaluate(
          ".//text()[normalize-space(.)!='']",
          document.body,
          null,
          XPathResult.ORDERED_NODE_SNAPSHOT_TYPE,
          null
        ),
        i,
        textNode,
        cnt,
        out,
        j,
        pfx,
        sfx;
      for (i = 0, l = xPathResult.snapshotLength; i < l; i++) {
        textNode = xPathResult.snapshotItem(i);
        if (
          textNode.parentNode.nodeName.toLowerCase() == "script" ||
          textNode.tagName == "style"
        )
          continue;
        cnt = textNode.data.split(/\s/g);
        out = [];
        for (j = 0; j < cnt.length; j++) {
          if (cnt[j].replace(/[\s]/g, "") == "") continue;
          pfx = !Math.floor(Math.random() * 10)
            ? ""
            : prefixes[Math.floor(Math.random() * prefixes.length)];
          sfx = !Math.floor(Math.random() * 10)
            ? ""
            : suffixes[Math.floor(Math.random() * suffixes.length)];
          out.push(pfx + cnt[j] + sfx);
        }
        textNode.data = " " + out.join(" ") + " ";
      }
    })();
```

Long words can make short work of your design

Fake Germanizer

```yaml
---
slug: developing-react-native-with-expo-and-flipper
published: 2021-12-07T01:00:00.000Z
description: Goodbye Chrome Debugger, hello Flipper
repost:
  url: https://blog.expo.dev/developing-react-native-with-expo-and-flipper-8c426bdf995a
  text: This article was written for the Expo Blog and appeared with the title _Developing React Native with Expo and Flipper_. It's been reproduced here with permission.
changelog:
  - on: 2021-12-03
    note: There’s now an Expo community plugin for setting up Flipper in your Expo projects. The section on Flipper integration was updated to reflect this change.
---
```

There’s a lot of great React Native modules out there, but most of them require a custom iOS or Android build, leaving “ejecting” as the only option for Expo users. That changed in SDK >42, when Expo added the ability for Expo Application Services to create custom development builds of your app.

What’s a custom development build? It’s not just any Expo Go app, it’s your Expo Go app; complete with any React Native module you ever wanted, all without ever having to hit the eject button.

And now that Expo 43 includes React Native 0.63, [Flipper by Facebook](https://fbflipper.com/) is available by default; there’s no better time to ditch the old “Chrome Developer Tools” way of debugging your React Native app. Not only will you gain access to device information and crash logs, you’ll also have Flipper’s amazing plugin library. Everything from Apollo to Redux debuggers are in there.

If you’d like to skip to the end result, check out the [jakobo/expo-cdc-example](https://github.com/jakobo/expo-cdc-example) repo as that’s what we’ll be building towards.

## Overview
- Configuring your code for EAS
- Creating a development build of your app
- Connecting to your app via Flipper
- Final considerations

## Create an EAS-Friendly Expo App
We’re going to start our journey by making a run-of-the-mill Expo application (install Expo CLI if needed with `npm install -g expo-cli`). When we run `expo init -t expo-cdc-example`, we’ll be given a standard Expo application, configured for Typescript. If you would prefer vanilla JavaScript, just omit the `-t` option. The result (minus our hidden folders and our lockfile) looks like this:

```
/
|- assets
|- app.json
|- App.tsx
|- babel.config.js
|- package.json
|- tsconfig.json
```

From here, we can run `expo start` and view our Expo app. We could also build locally, but what we’re really looking for is to let someone else do all the work for us. This is doubly true if we’re developing on Windows, as XCode isn’t even an option for making an iOS app. Enter Expo Application Services (EAS).

EAS lets you use Expo’s hardware to build your iOS/Android application, and has its own CLI. You can use EAS for building non-expo applications as well, so the EAS cli isn’t bundled with Expo. We’ll need to install it globally like we did with our `expo-cli`.

    npm install -g eas-cli

Let that run, and you’ll have a new `eas` command available. The EAS CLI doesn’t have an `init` command, and while running the `eas` build command will generate the configuration file for us, we’ll have greater control if we create the file ourselves. Create a file named `eas.json` in your application root and copy/paste the following configuration in:

```json
{
  "cli": {
    "version": ">= 0.34.0"
  },
  "build": {
    "base": {
      "env": {}
    },
    "development": {
      "extends": "base",
      "developmentClient": true,
      "distribution": "internal",
      "releaseChannel": "default",
      "android": {
        "buildType": "apk"
      }
    },
    "production": {
      "extends": "base",
      "distribution": "store",
      "releaseChannel": "stable",
      "android": {
        "buildType": "app-bundle"
      }
    }
  }
}
```

This sets up EAS to build something we’ll call a “preview” build. It will behave just like our production app, but it’s claimed ownership of the default release channel. By utilizing `releaseChannel: "default"` in our development build, and not our production builds, we can make sure that an accidental `expo publish` doesn’t push code to our live application. There’s [a lot more to `eas.json`](https://docs.expo.dev/build/eas-json/) than we’re covering in this article, but the above is enough for us to launch our builds. If you’re already using the default channel for your app store releases, you can change value of the `releaseChannel` property.

We can now tell EAS to build our app. We’ll want to specify our build profile preview as part of the command.

If this is your first time running EAS, you’ll want to associate any iOS devices with your ad hoc provisioning profile. You can [follow the steps in the Expo documentation](https://docs.expo.dev/build/internal-distribution/), or just run `eas device:create` and follow the prompts. To generate our build, we just need to run `eas build --profile=preview`.

EAS will be off building things, and you can go get a cup of coffee. ☕

After a while, you’ll have a shiny build that you can install on your iOS/Android device! EAS runs in the background too, so no need to wait for a build to finish, you can get back to coding.

## Configuring EAS For a Development Build

So far, we’ve created an Expo app, we’ve configured EAS, and we’ve kicked off a custom build. But EAS doesn’t just let you make custom production builds, it **lets you make development builds too!** That’s right, it is your own Expo Go, with your own modules. And we’ll be using EAS to build it.

Enabling a development builds means creating a new profile in our `eas.json`. These additions should go alongside your preview build.

```json5
{
  "build": {
    // ...
    "devclient": {
      "extends": "base",
      "releaseChannel": "default",
      "distribution": "internal",
      "developmentClient": true,
      "android": {
        "buildType": "apk"
      },
      "ios": {
        "simulator": true
      }
    }
  }
}
```

Our new profile, `devclient` adds a few EAS options. The `developmentClient` flag tells EAS that we want a custom build of the Expo development app, and the `ios.simulator` field tells EAS not to create an iOS app, but a bundle that’s designed for the iOS Simulator.

Now, kick off your new build and go get more coffee. ☕

    eas build --profile=devclient

When that finishes up, you’ll have a custom simulator build. When you install that on your simulator and launch it, it will look different than your usual Expo Go app.

![The Expo Custom Development Client launch screen](https://user-images.githubusercontent.com/1795/146845551-1bd86cb8-8236-4805-ba10-2bf24ab0cb82.png)
_The Expo Custom Development Client launch screen_

Instead of the usual expo start command, it tells us our app needs to be started with the `--dev-client` flag, so let’s do that. Stop your expo start if it’s running, and restart it with `expo start --dev-client`.

The project will show up in your development build with a URL structure of `exp+<project>://expo-development-client/…`. The `exp+<project>` protocol helps the native OS tell the difference between your custom Expo app and any others you might have on the simulator or device.

So long, Expo Go, hello Development Builds. 💅

## Hello Native Modules, Hello Flipper

All of the above setup needs to be done exactly once, and the payoff is huge. We’re going to get a better debugger for React Native, called [Flipper](https://fbflipper.com/), by Facebook. Go download the app, and we’ll set up our development build to use this native module.

First, we’ll need to install Flipper and its dependencies, which is a straightforward pair of command: `expo install react-native-flipper react-devtools-core` for our expo related code and `yarn install expo-community-flipper` for our build plugin we'll be using later.
Then, we’ll need to tell our App to connect to Flipper in our `App.tsx` file:

```tsx
// App.tsx
import { connectToDevTools } from "react-devtools-core";
if (__DEV__) {
  connectToDevTools({
    host: "localhost",
    port: 8097,
  });
}
```

Since we’ve added a new native module, we’ll need to create a new development build that has our native iOS and Android code bundled in. Because Flipper also requires changes to our app’s `Podfile`, we’re also going to make use of Expo’s [Config Plugin](https://docs.expo.dev/guides/config-plugins/) system. Because we already installed the [community Flipper plugin](https://www.npmjs.com/package/expo-community-flipper), it's as easy as adding the plugin to our `app.json`.

```json5
{
  "expo": {
    // ...
    "plugins": ["expo-community-flipper"]
}
```

One last time (and one last ☕): `eas build --profile=devclient`

Install the new build on your simulator, open up the Flipper App and it just works 🤯

You can repeat this for any Native Module you want, though Flipper is an excellent starting point since it lets you use `react-native-reanimated` v2 with TurboModule support, frees you from the built-in Chrome debugging tools, gives you crash logs, and more. Just remember that you’ll need to create a new development build when you add a new module, or make judicious use of try/catch blocks to ensure you don’t throw an error.

## Keep Building From Here

This is just the start of what you can do with Expo’s development builds and Native Module support. Most Native Modules will work out of the box in a development build thanks to React Native’s autolinking, but if you do need custom changes to your `Info.plist` or `Manifest.xml` files (or even editing a `Podfile` like we did today), Expo can take care of that too as part of the build process. These [config plugins](https://docs.expo.dev/guides/config-plugins/) let you modify the plist and manifest without ejecting your app and are configured from within your `app.json`.

If you still run into issues, there’s an #EAS channel on the [Expo discord](https://discord.gg/4gtbPAdpaE) and the [Expo Forums](https://forums.expo.dev/) have an entire section dedicated to EAS.

There’s now an Expo community plugin for setting up Flipper in your Expo projects. The section on Flipper integration was updated to reflect this change.

This article was written for the Expo Blog and appeared with the title _Developing React Native with Expo and Flipper_. It's been reproduced here with permission.

{
  "build": {
    // ...
    "devclient": {
      "extends": "base",
      "releaseChannel": "default",
      "distribution": "internal",
      "developmentClient": true,
      "android": {
        "buildType": "apk"
      },
      "ios": {
        "simulator": true
      }
    }
  }
}


// App.tsx
import { connectToDevTools } from "react-devtools-core";
if (__DEV__) {
  connectToDevTools({
    host: "localhost",
    port: 8097,
  });
}


{
  "expo": {
    // ...
    "plugins": ["expo-community-flipper"]
}


Developing React Native with Expo and Flipper

```yaml
---
# This frontmatter will be used by CodeDrift
slug: reparenting-iframes
published: 2015-03-17T00:00:00.000Z
description: How to change an iframe parent node without a reload
changelog:
  - on: 2017-10-04
    note: It's been noted in [comments](https://medium.com/@mbogdan/even-in-webkit-support-for-the-so-called-magic-iframe-has-been-removed-882a27d1bf15) and confirmed that the reparenting with `adoptNode()` no longer works and will instead cause the iframe to reload. Based on the [bug report](https://bugs.webkit.org/show_bug.cgi?id=13574), we're also not likely to see this situation change either. So until the situation changes, the answer is "no, you can't call `adoptNode()` without the iframe reloading. You will likely need to get clever with positioning items absolutely.
  - on: 2020-02-23
    note: This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal). As a result, many links in this piece were left pointing to archive.org destinations.
---
```

If you're playing around with iframes, there's a nasty bug that occurs when you attempt to move the iframe using methods such as `appendChild` or `insertBefore`. How nasty the bug is depends on how tolerant your code is of reloading behaviors. Any attempts to move the iframe around will cause reloads in older Safari & Firefox versions. The problem lies in how the browsers interpret these events, removing the node and then adding it to its new destination which triggers a reload. If you're trying to move iframes around on the page with something like a drag and drop interface, this rears its ugly head. Strangely enough, IE doesn't exhibit this behavior, which leaves the problem to just Webkit and Firefox.

## Workarounds in CSS

A quick search reveals only one consistent technique for moving iframes about on the page using CSS- positioning absolutely. An absolutely positioned iframe can be readjusted easily, and can track directly on top of a placeholder such as a div. This works pretty fantastically in all but the most complex of layouts where absolute positioning requires a relative parent in just the right place.

```html
<body>
  <div id="#placeholder"></div>
  
  <iframe id="theFrame" src="http://www.example.com" style="left: -12345px; top: 0; position: absolute;"></iframe>
</body>
```

```js
(function () {
  var div = document.getElementById("placeholder");
  var iframe = document.getElementById("theFrame");

  window.setInterval(function putFrameOntoPlaceholder() {
    // 1) measure the size of the iframe, change size of div
    div.width = iframe.offsetWidth + "px";
    div.height = iframe.offsetHeight + "px";

    // 2) get the div's position on the page, position iframe
    iframe.left = div.offsetLeft + "px";
    iframe.top = div.offsetHeight + "px";
  }, 100);
})();
```

In layouts that make heavy use of relative positioning, it's easy to see where this might run into snags. Styles on either the div or iframe could mess up the alignment a bit, and while it's correctable it's not very feasible for a large scale site where there's going to be several developers building code.

## Webkit & Gmail: Adopting Nodes

In July 2010, Google silently released an improvement to Gmail. It was most noticeable when using the gmail chat feature. You could open the chat window, "pop" it out of the main window, and then *close the parent window* and everything continued to work. The most fascinating part about this was the minimal load time in the popup. It was as if the entire Gmail environment were being ported over to the popup as part of the unload events. That's exactly what's happening. **The iframe contained the code both the main page and popup needed.**[The initial webkit bug on reparenting iframes](https://bugs.webkit.org/show_bug.cgi?id=32848) became the ideal fix.

This worked thanks to the [DOM Level 3](http://www.w3.org/TR/DOM-Level-3-Core/core.html) core; a new DOM manipulation method was added called `adoptNode`. Adopting an HTML node would simply move the node and in the case of the iframe, would not trigger a reload of the content. Coming back to Gmail, the JavaScript needed for the popup window is loaded into an iframe. When the parent window is closed, `adoptNode` is called, moving all of the necessary code into the popup window before itself closing.

## And Firefox? Opera? Other Browsers?

Unfortunately, as of this writing, Firefox still has [an open ticket on the [broken adoptNode behavior](https://bugzilla.mozilla.org/show_bug.cgi?id=254144), with no fixes in sight. As a last ditch effort, we can fall back to `importNode`, we will just take the performance hit and trigger an undesirable reload which can possibly erase state within the iframe if there is any.

This stuff hasn't been validated against Opera. Since Opera is based on webkit, it shouldn't be exhibiting this bug.

## Falling Back

Using the below code, it's possible to use `adoptNode` when it's available, and then fall back to the `importNode` as a last resort. Depending on the browser, you may still reload the sub-document. Should the `adoptNode` bug ever get fixed, you'll get a large performance boost in iframes and popups.

```js
try {
  document.adoptNode(window.opener.$("#my-iframe"));
} catch (exc) {
  // performance hit. Either importNode, or load the
  // frame contents with appendChild()
  // remember you'll also need state somewhere
  // in window.opener
  document.importNode(window.opener.$("#my-iframe"));
}
```

It's been noted in [comments](https://medium.com/@mbogdan/even-in-webkit-support-for-the-so-called-magic-iframe-has-been-removed-882a27d1bf15) and confirmed that the reparenting with `adoptNode()` no longer works and will instead cause the iframe to reload. Based on the [bug report](https://bugs.webkit.org/show_bug.cgi?id=13574), we're also not likely to see this situation change either. So until the situation changes, the answer is "no, you can't call `adoptNode()` without the iframe reloading. You will likely need to get clever with positioning items absolutely.

How to change an iframe parent node without a reload

(function () {
  var div = document.getElementById("placeholder");
  var iframe = document.getElementById("theFrame");

  window.setInterval(function putFrameOntoPlaceholder() {
    // 1) measure the size of the iframe, change size of div
    div.width = iframe.offsetWidth + "px";
    div.height = iframe.offsetHeight + "px";

    // 2) get the div's position on the page, position iframe
    iframe.left = div.offsetLeft + "px";
    iframe.top = div.offsetHeight + "px";
  }, 100);
})();


try {
  document.adoptNode(window.opener.$("#my-iframe"));
} catch (exc) {
  // performance hit. Either importNode, or load the
  // frame contents with appendChild()
  // remember you'll also need state somewhere
  // in window.opener
  document.importNode(window.opener.$("#my-iframe"));
}


Reparenting iframes

```yaml
---
slug: window-postmessage-and-ie
published: 2015-03-14T00:00:00.000Z
description: Limitations on a crucial cross-origin API
changelog:
  - on: 2016-01-12
    note: IE 9 Support has **ended**. That means we can finally just use `window.postMessage` the way it was intended. You can safely discard 99% of this now!
  - on: 2020-02-23
    note: This was rescued from the archives of the old [felocity.org](https://web.archive.org/web/20050507121849/http://www.felocity.org/journal). As a result, many links in this piece were left pointing to archive.org destinations.
---
```

It started out innocently enough: the LinkedIn Platform needed to run our Professional Plugins on third party sites. To do the basic process of resizing iframes (to fit our plugins), we needed a way to talk across domains. With HTML5, we gained access to `postMessage`, which was a nice standard way of talking between domains. For the browsers such as IE6 and IE7, libraries such as [Apache Shindig](http://shindig.apache.org/) and [easyXDM](http://easyxdm.net/) have found a way to use Visual Basic objects (and later Flash) to coerce older browsers into supporting a compatible interface. Internet Explorer 8 and 9, however, have a set of quirks in its `window.postMessage` which will frustrate users of home-grown solutions and libraries alike. Inevitably, another developer is going to run into these problems.

## Internet Explorer Recasting Things to "Object"

If you're using popup windows alongside a cross domain library, you'll probably be calling methods in `window.opener` to avoid having to create multiple listeners for the postMessage object. While this nuance isn't postMessage specific, if you're using an RPC implementation in a library, you can expect your `success()` and `error()` methods to get mangled.

```js
// child window
function childFunction() {}
window.opener.parentFunction(childFunction);

// parent window
function parentFunction(f) {
  alert(typeof f); // object
}
```

To get around this, you can either serialize the function using or make sure that `parentFunction` doesn't depend on a `typeof` check. Even though in the parent window it looks like an object, it will still begin with "function" when serialized. As an added check, you can verify the existence of `call()` and `apply()` before treating it like a callable object. On some level, what Internet Explorer is doing is correct, as Functions are Objects just like everything else. Since the Parent and Child window both have different window objects, their Function objects would also be different, because reasons.

```js
// Oh IE...
window.opener.Function === Function; // false
```

As of Internet Explorer 9, the typeof operator properly displays the object as Function, implying it's callable. Therefore, the fix can be localized to IE 6/7/8 using a conditional comment in JavaScript.

## No Communication Across Popup Windows

If you're adventurous and forgo a library for cross domain communication, do not expect it to talk to popup windows in Internet Explorer 8. The worst part about this is that it's a [known bug in postMessage](http://blogs.msdn.com/ieinternals/archive/2009/09/16/Bugs-in-IE8-support-for-HTML5-postMessage-sessionStorage-and-localStorage.aspx) for which the current solution is to make an iframe, for which you can then hand off the `window.opener` assignment. There are two problems with this approach. In order to reach our opener, the popup window and iframe will need to be on the same domain or set their `document.domain` property. The second problem is a bit more vexing, as reaching through `window.opener` triggers the first bug mentioned above. The workarounds are still usable, but if you're passing anything more complex than a primitive such as a string between windows, you're sunk.

To be safe, it's worth wrapping the things in postMessage as a string using JSON. If you need to persist objects between windows, giving objects a way to instantiate on the other side is really the only option.

## Origin Mismatching

The rules for postMessage security are pretty straightforward. When calls are made, the origin is derived from the window object used to initiate the postMessage call. The below code highlights the IE8 inconsistency. For whatever reason, reaching through `window.opener` again causes issues, and the domain is changed to the initial script entry point into postMessage versus the page where the code actually resides.

```js
// popup window (popup.example.com)
document.domain = "example.com";
window.opener.postMessageInterface("string");

// window that opened the popup (iframe.example.com)
// just contains normal postMessage()
document.domain = "example.com";
function postMessageInterface(str) {
  window.top.postMessage(str);
}

// On the ultimate listening page, IE8 identifies the origin
// as where the call stack began. Other browsers all identify
// the origin as the domain from which the .postMessage() call
// was made
window.addEventListener(
  "message",
  function receive(evt) {
    // IE 8
    alert(evt.origin); // popup.example.com

    // Every Other Browser
    alert(evt.origin); // iframe.example.com
  },
  false
);
```

The workaround for this is that we can use `setTimeout`. By putting the intent to call postMessage back on the stack, we are able to resolve it with the correct origin.

```js
function postMessageInterface(str) {
  window.setTimeout(function () {
    window.top.postMessage(str);
  }, 0);
}
```

The need to make use of this workaround is even more important if you're using `document.domain` due to a CDN. Even though it's been set in the example, the values are ignored for `postMessage` on IE (even though those exact same rules allowed `window.opener` to be reached in the first place).

## Just Use easyXDM

This is a long post for saying [easyXDM](http://easyxdm.net/) has already solved these problems if you need to support IE 8 and IE 9. We're actively working towards a world where you won't need the library. But if you do, it's there.


IE 9 Support has **ended**. That means we can finally just use `window.postMessage` the way it was intended. You can safely discard 99% of this now!

Limitations on a crucial cross-origin API

// child window
function childFunction() {}
window.opener.parentFunction(childFunction);

// parent window
function parentFunction(f) {
  alert(typeof f); // object
}


// Oh IE...
window.opener.Function === Function; // false


// popup window (popup.example.com)
document.domain = "example.com";
window.opener.postMessageInterface("string");

// window that opened the popup (iframe.example.com)
// just contains normal postMessage()
document.domain = "example.com";
function postMessageInterface(str) {
  window.top.postMessage(str);
}

// On the ultimate listening page, IE8 identifies the origin
// as where the call stack began. Other browsers all identify
// the origin as the domain from which the .postMessage() call
// was made
window.addEventListener(
  "message",
  function receive(evt) {
    // IE 8
    alert(evt.origin); // popup.example.com

    // Every Other Browser
    alert(evt.origin); // iframe.example.com
  },
  false
);


function postMessageInterface(str) {
  window.setTimeout(function () {
    window.top.postMessage(str);
  }, 0);
}


window.postMessage and Internet Explorer

```yaml
---
slug: tailwind-css-in-react-native
published: 2021-05-03T00:00:00.000Z
description: Tailwind's CSS classes in your React Native projects
changelog:
  - on: 2021-12-20
    note: I'm currently using [twrnc](https://www.npmjs.com/package/twrnc) which is an evolution of the tailwind-rn library. A pattern like `gust` is great for custom components, but right now, the React Native ecosystem almost always needs to pass style objects around.
---
```

React Native is very behind the web when it comes to UI libraries. There’s two likely reasons for this: CSS on the web is easy and the object known as `StyleSheet` in React Native is not.

At a fundamental level, the syntax of `StyleSheet` causes a lot of problems. It looks like CSS (the kind you’d use in React’s `style` prop), it acts like CSS (most of the time) just like you’d expect in a browser, but then you encounter one of dozens of scenarios where React Native takes the CSS specification and yeets it out the nearest window.

- Shadows are a custom object syntax instead of a conversion of `box-shadow`
- Random elements such as `ScrollView` introduce custom non-standard props such as `elevation`, but also only for specific platforms
- You get flexbox. Only flexbox. Everything flexes. Everything is boxes. Except for the `flex` property which has decided it is not actually `flex` as you’d expect but just a number that translates to a percentage. But not a useful number-to-percentage like `55` meaning `55%`. Instead, you get an alternate universe number that is expressed as a ratio based on the sum of all flex properties on sibling nodes. It’s a universe where `flex:1` means both `100%` and `50%` based on elements outside of its rendering tree. /rant

Yes, a lot of this will be fixed over time as React Native continues to improve on its style interpreter, but we are still converting from CSS to platform-specific layout calls. I don’t think we’ll ever reach parity, but I’d also like to avoid the programming equivalent of “hammering my thumb” every time I lay things out in a mobile app.

The truth is, what I want is something as easy as [Tailwind CSS](https://tailwindcss.com/) in React Native. If you haven’t tried Tailwind yet, you should. It’s trivial to make something look good, and it’s adaptable enough withstand design changes. In React Native, there's a few really solid options:

* **tailwind-rn**([github](https://github.com/vadimdemedes/tailwind-rn), [npm](https://www.npmjs.com/package/tailwind-rn)) is the OG of Tailwind in your React Native projects. It works by converting the Tailwind styles into JSON, which is memoized in your app on access. Because it tries to be only a translation layer between systems, complex behaviors such as breakpoints and React Native's Appearance are not supported out of the box.
* **tailwind-rn-gust** (_archived_ [github](https://github.com/jakobo/tailwind-rn-gust), [npm](https://www.npmjs.com/package/tailwind-rn-gust)) extends the `tailwind-rn` library by adding utilities for custom component creation. By introducing a React Component generator, it's possible to add additional props to turn "on" flags inside of a call to `tailwind-rn`'s style generator. _However, I wouldn't recommend this, as the components add an additional layer of complexity once you want to use `Animated.View` or other 3rd party components that want to leverage the `style` prop for stylistic behavior.
* **twrnc** ([github](https://github.com/jaredh159/tailwind-react-native-classnames), [npm](https://www.npmjs.com/package/twrnc)) Is what I'm currently using and it leverages [Tailwind's JIT Compiler](https://v2.tailwindcss.com/docs/just-in-time-mode) to remove the built steps inherent in other Tailwind-bridging systems. Out of the box, it uses React Native's hooks for figuring out screen sizes and appearance, making it easier to use normal Tailwind prefixes such as `lg:text-xl`.

Originally, this post focused on `tailwind-rn-gust`, but I think it's becoming more clear that until there's an official Tailwind for React Native, `twrnc` is going to be the truest-to-Tailwind experience you'll find on the platform.

Instead, here's how you'd accomplish something like a rich "press" interaction using `twrnc`. (You can also put both styles into a single `tw.style` call, but I find it easier to read leveraging the array syntax supported for multiple style objects.)

```jsx
<MyComponent style={({pressed}) => [
  tw`bg-red-500`,
  tw.style({
    "bg-blue-500": pressed
  })
]} />
```

The best thing about the `tw` helper is that it's built for generating `style` objects, meaning it'll work with Moti, Reanimated 2, and many other component libraries that focus on the `style` prop.

## Gust Archived
And so, with great alternatives out there, `gust` is archived. The [entire library](https://github.com/Jakobo/tailwind-rn-gust) is only 159 lines and is easy to port into your own existing React Native project if you need the features. However, until we see something like a native `className` in React Native, the `style` tag is going to be the default operating mode for rich styles, animation, and state based effects.


I'm currently using [twrnc](https://www.npmjs.com/package/twrnc) which is an evolution of the tailwind-rn library. A pattern like `gust` is great for custom components, but right now, the React Native ecosystem almost always needs to pass style objects around.

Tailwind's CSS classes in your React Native projects

<MyComponent style={({pressed}) => [
  tw`bg-red-500`,
  tw.style({
    "bg-blue-500": pressed
  })
]} />


A Guide to Tailwind CSS in React Native

```yaml
---
slug: time-based-queues-serverless-docmq
published: 2022-08-31T12:39:00-07:00
description: Nerding out on time-ordered priority queues
changelog: ~
---
```

Building a [serverless-friendly queue system](https://taskless.io) continues to be an adventure. Recently, we had to say goodbye to our Redis queues and replace them with something more durable. The journey there involved a lot about time storage, visibility windows, and how the most popular queue solutions solve these problem. So, if you were curious what makes Taskless reliable, the short answer is a database optimized for time-based queries. For a longer answer, we need to go on a journey.

## First, it's Time for Events

The first rule of the JAMStack is that everything is an event. Because servers aren't standing by and waiting for your every request, events are the only way for a serverless provider to know your code needs to run. In practical terms, when you make a request to a Next.js app or AWS lambda, that small chunk of code is invoked in response to the URL request, executed, sends the reply, and then shuts down. This is well and good for operations like "forward this blog post to a friend" or "leave a comment about how you definitely wouldn't forward this blog post to a friend," given both examples begin with an obvious interaction. But what about an operation like "send this to my inbox tomorrow morning"?

It turns out, time ruins everything.

Time sucks, and if you don't believe me, go [read Zack Holman's excellent piece on the matter](https://zachholman.com/talk/utc-is-enough-for-everyone-right) and return once you've come to a similar conclusion. Now that we agree the concept of time was a mistake, let's talk about sending something to your inbox "tomorrow". To avoid headaches, we're going to make some poor assumptions about what tomorrow means. In Luxon, we can express this arbitrary definition of "tomorrow" as `DateTime.now().zone("America/Los_Angeles").plus({days: 1}).startOf("day").set({hour: 8})`. Bam. 8am "tomorrow" at of this exact moment. All we need to do is "event" at that time, and we're set!

## Store the Time in the Queue Itself - aka The SQS Experiment

The purest of serverless ambitions would be to never persist anything under our ownership. To that end, [Amazon SQS](https://aws.amazon.com/sqs/) and lambdas are an excellent fit. The platform has limitations, but those could be worked around with a bit of duct tape.

First, the maximum delay on a lambda is 15 minutes, so SQS would need to send the job to a lambda at the end of the maximum delay, which would only report success if it could requeue the job for another 15 minutes. We'd repeat this ad-nauseum until we reached our magical "8am tomorrow" requirement and then run our job.

This worked really well (and was insanely cheap). Each job would require a maximum of 96 SQS calls per day in a pending state, and it's only $0.50 per million requests. That's 10,000 jobs per month, delaying every job up to a maximum of one day, with just AWS tools. In practice, the system has a lot of opportunity to fail. Specifically 672 opportunities for every day the job sits in SQS and must be constantly requeued,

Honestly, if SQS didn't have an arbitrary 15 minute delay, we'd be done. But it doesn't so we're not.

## Tables, Indexed by Time

It turns out that databases are really good at time stuff, and not just because [ISO-8601](https://en.wikipedia.org/wiki/ISO_8601) is both readable and easy to sort by. But also because really smart people made time (and its storage) one of the primitives in their db systems.

Databases are also really good and indexing non-time things. So imagine you have a table

| column | type |
| :--- | :--- |
| id | any |
| failed | boolean = false |
| visible | time, nullable |
| deleted | time, nullable |
| ack | any, nullable, unique (on non-null) |
| payload | text, nullable |

Now it's easy to describe the operations in order to take, process, and report on jobs.

* Your next jobs are at `visible <= now() AND deleted IS NULL`
* Your failed jobs are at `deleted IS NOT NULL AND failed = TRUE`
* Your completed jobs are at `deleted IS NOT NULL AND failed = FALSE`
* You can retry a job by changing its `visible` time
* You can complete/fail a job by setting `deleted` and `failed` in the same transaction
* A unique `ack` value lets you find a single job in the table so that you are complete/fail/retrying only one specific job

As an added bonus, using just-a-database means that we have separated our queue (what do we need to run) from our worker (the actual running). The pattern was simple enough that I bundled it up as [DocMQ](https://github.com/jakobo/docmq). It has a persistent worker if that's something you want, or you can just run DocMQ for adding jobs to your queue database and process them on your own time.

## Time to Return to Events

The beautiful part about using a database is that once it's in the database, it's _there_. We don't need persistence unless we want it, and polling is good enough for most use cases. All we need to do is poll in response to an event; that event is _time itself changed_. 🤯

Stick with me here. Imagine if we have a cron job via [Netlify Scheduled Functions](https://docs.netlify.com/netlify-labs/experimental-features/scheduled-functions/) or [lambda cron](https://docs.aws.amazon.com/lambda/latest/dg/services-cloudwatchevents-expressions.html) that runs once per minute (or per-second if you'd like to YOLO). A single query tells you what capacity you need:

```sql
SELECT count(*) FROM your_table WHERE visible <= now() AND deleted IS NULL
```

You get 20 rows back, spin up 20 serverless functions, each to claim 1 row:

```sql
UPDATE your_table SET ack = gen_random_uuid(), visible = visible + interval '30 seconds'
WHERE visible <= now() AND deleted IS NULL
LIMIT 1
```

💥Instant scalability. If you need more granularity, tighten the cron. If you need sub-second control then you probably do want a server. Real talk though, if you're sending emails the latency isn't your job system.

## How This Relates to Taskless

At [Taskless](https://taskless.io) we ran on the cron job / polling design until we were invoking an unreasonable number of serverless functions on the minute and customers wanted sub-minute responses. Since our scheduled function infrastructure didn't let us go to per-second, we switched to persistence. In short, we ran the servers so you could go 100% serverless. 🎉

Because [DocMQ](https://github.com/jakobo/docmq) is on GitHub, you can replicate it yourself. There's even a [postgres prototype](https://github.com/jakobo/docmq/tree/labs-postgres) that will get merged down soon in case you're a fan of Neon, Supabase, or other serverless-friendly postgres instances.

### Some Quick Answers

Added because these came up in casual conversation

* Does the DB matter? - No, as long as your DB understands time
* Biggest drawback? - Performance. A relational db will be slower than an in-memory solution in almost every use case. Unless you're building a real-time application such as chat, the latency of a DB touching disk is not going to be why things are slow
* What DB does Taskless use? - We use MongoDB Atlas currently. We like their time series support



Nerding out on time-ordered priority queues

thead

tbody

table

SELECT count(*) FROM your_table WHERE visible <= now() AND deleted IS NULL


UPDATE your_table SET ack = gen_random_uuid(), visible = visible + interval '30 seconds'
WHERE visible <= now() AND deleted IS NULL
LIMIT 1


Designing A Time-Based Queue for Serverless

```yaml
---
slug: nodejs-memory-leak-heroku
published: 2022-07-19
description: You can connect to your production Heroku instance, debug it live, and even trace down memory leaks.
keywords: heroku, r14, r15, memory leak
changelog: ~
---
```

There's nothing more frustrating that your node.js app leaking memory. If you're coming here because of Heroku R14 / R15 errors, take some deep breaths, it'll be okay. Heroku's smart enough to swap to disk when this happens, and will kill and restart your app if things get absolutely wild.

## Background

![Heroku Dashboard with a lovely orange band of R14 errors](https://user-images.githubusercontent.com/1795/163650457-2371246f-bc80-4482-b42e-8bfdbf664a1a.png)

It started with my email box getting pelted with warnings that the memory quota on a project I advise for was getting hit with R14 errors. Going to the Heroku metrics page confirmed that yes, something was clearly wrong. In the above picture, you can see again and again the memory usage surge and then Heroku responding with an R15 and force rebooting the server. Moments later, the memory usage was growing again.

It's a Memory Leak, Ruin of Productivity and afternoons. 😭 

However, since the last time I dealt with a memory leak (2011) the tools have come a long way. So here's how you triage a production memory issue, run analysis against your currently leaking production boxes, and stamp out the issue.

## Modern Tools

The tools for dealing with a memory leak have gotten much better, especially if you're on Heroku. You only need two:

* A current version of Chrome (which comes with the amazing v8 debugger)
* The Heroku CLI (if on Heroku)

The first does 90% of the heavy lifting, while the second now has all the commands we need to turn on a debugger in production and forward debug data to our local machine. Make sure you've logged in to the Heroku CLI if you're triaging Heroku.

## Opening Chrome's Inspector

The Chrome Developer Tools are always available on `chrome://inspect`, which can be put in your browser's address bar. The DevTools are useful for debugging everything from node.js processes to Service workers. For now, we want to stay on the `Devices` tab, make sure `Discover network targets` is checked, and leave that window open while we set other things up. You may even see a few device targets show up that Chrome's discovered. In my case, apparently my TV can be seen. Odd, but neat.

![The Chrome developer tools](https://user-images.githubusercontent.com/1795/163651415-d75abf95-e781-45fb-ab2c-3e9adcb6ba1d.png)

## Enabling the Debugger With a Signal

The most common way to start debugging node is to add the `--inspect` CLI arg, either directly (via `node --inspect <your file>`), or in the appropriate env values ( `NODE_OPTIONS="--inspect" node <your file>`). In production environments, it may not be desirable to enable the debugger on every start; in the case of Heroku which randomly assigns ports, you can't even guarantee the debug port of `9229` is available. But, you can enable the debugger a different way.

From the [node.js debugging page](https://nodejs.org/en/docs/guides/debugging-getting-started/)

> Node.js will also start listening for debugging messages if it receives a SIGUSR1 signal. (SIGUSR1 is not available on Windows.) In Node.js 7 and earlier, this activates the legacy Debugger API. In Node.js 8 and later, it will activate the Inspector API.

Sending a signal is as easy as using the unix `kill` command, where `<PID>` is your node.js process ID:

```sh
$ kill -usr1 <PID>
```

You can try this locally by running `ps aux | grep node` to find your node.js process, and then using the `PID` for the above `kill` command. Personally, I find that the `usr1` signal is not only more reliable (works everywhere) but also means you don't have to completely restart the server. When you send `usr1`, node just sees the signal and starts up its debugger. It'll pop up moments later in the Chrome DevTools.

![local development showing the inspector starting](https://user-images.githubusercontent.com/1795/163651616-5d423ec2-bdd9-4bcd-828c-29e4bbf5335e.png)

![the development tools showing the new remote target](https://user-images.githubusercontent.com/1795/163651579-b7181e72-86d9-47e9-9d46-eb4c3cf76bc4.png)

Hit the `inspect` link and you'll be taken to a very familiar DevTools interface, but optimized for node.js debugging. In addition to the Console and Sources, you'll also have "Profiler" for CPU profiling, and "Memory" for memory usage profiling,

![connected development tools, ready to go](https://user-images.githubusercontent.com/1795/163651715-22c7a39a-81cb-43da-bc97-4e17523ffb56.png)

## Connecting to Heroku in Production

Once that's all working, it's time to connect to our production instance in Heroku and take some live samples. Close the inspector and local development server if needed, and make sure your `heroku` command works. If you're using a different environment such as Digital Ocean, look up SSH instructions to get to your host. They'll be different on every provider.

For heroku, there's the `ps:exec` command which gives you a shell on your default process for dyno 1. Be sure to specify your `<APP_NAME>` so that you're ssh-ing into the correct box, and if you need to connect to a dyno process that's not `web`, add the `-d <dyno>` flag.

```sh
$ heroku ps:exec -a <APP_NAME>
```

![connecting to a remote box over SSH](https://user-images.githubusercontent.com/1795/163652037-8309d4c9-452a-4bf5-848a-98f466346471.png)

From here, we can search our node process with `ps aux | grep node` and start the debugger with `kill -usr1 <PID>`. Our debugger will start, but obviously we can't see it yet. Heroku (and other providers) don't expose port `9229` to the internet for security reasons. Open another terminal window and use heroku's forwarding command to create a local tunnel for your debugger (and add the optional `-d` flag if needed).

```sh
$ heroku ps:forward -a <APP_NAME>
```

And just like that, your production instance shows up in the Chrome DevTools.

## Sampling

Taking samples is straightforward. Wait for the VM instance size to grow, click the trash can icon (force GC) to ensure it's actually grown by a reasonable amount, and then hit the **Take Snapshot** button to capture a memory heap. Chrome DevTools will be busy for a bit, but it'll create a Heap, which it adds to the left-hand side of the Memory profiler. Every heap is a detailed analysis of what was in JavaScript at that exact moment. Every string, object, key, event, closure, callback; you name it, it's here.

Collect several samples (5-6 is ideal) since you'll be looking for identifiable trends. I also recommend waiting about 2-3 MB between each sample. We want our growth in memory to be noticeable, and whatever's leaking to stick out.

Here, I've got 6 production samples of a leaking app. In the time I sampled, we added about 15MB to the footprint. 😱 

![chrome devtools with several heaps loaded](https://user-images.githubusercontent.com/1795/163652520-5df538be-aa9a-4ae7-bf41-17617abe030f.png)

What matters is we can see the memory growing, we have several samples of the Heaps in question (which contain our growing memory), and those Heaps are readable by the Chrome DevTools. We can save the Heaps, disconnect from our production machine (which also closes the debugger), and get to our analysis.

## Using Heap Compare

Select the newest (largest) Heap, and above the list of objects in the heap, change from `Summary` to `Comparison`. This changes the Heap view from telling you what's in the Heap to what changed **between** Heaps.

![how to change from summary to comparison view](https://user-images.githubusercontent.com/1795/163652680-cd5f2530-6b12-4c00-8014-326df53c8a80.png)

Sort the table by `# Delta`, and the problem suddenly becomes very obvious.

![a sorted by delta comparison](https://user-images.githubusercontent.com/1795/163652782-5832abe0-c4c3-4e79-b804-100f1916fa47.png)

We can compare against a few other Heaps (that's why we captured them) to isolate the source of the memory leak. Here, I'm comparing Heap 5 to Heap 4, where the number of `(concatenated string)` is much smaller and easier to manage. There's a lot of weird date strings like "-137 days, 21" being created and retained. Clicking on one gives us the full Retainer log. It's part of a larger string, that's part of a larger string, that's in `discountTimeLeft` in the update cycle of a React component called `DiscountBanner`, inside of a... `Timeout`.

![The memory leak source](https://user-images.githubusercontent.com/1795/163653410-35a401c4-0b85-4eb6-9c14-4eec51878a0d.png)

Opening up the `DiscountBanner` culprit, there's a `setInterval` in the `constructor`. Fixing this and redeploying closed the memory leak. It turned out that when next.js was running in Heroku, the `setInterval` would just keep on running and building its string about how many days were left on the discount. Since that string was locked in the `setInterval` callback, it could never be garbage collected by v8. Add in a few thousand requests per second, and those discount strings would ultimately take up all of the available memory and cause Heroku to restart the server with an R15 error.

## More than Heaps

I only needed the Heap compare, but the CPU profiler and the Memory timeline (Allocation instrumentation on timeline) can help narrow the issue even further in a larger application. The hardest part is getting a reproducible issue. Looking at production not only makes it easier, but ensures you're solving the right problem. And when you're done, you'll never put a `setInterval` in a React component constructor again. 😆 

You can connect to your production Heroku instance, debug it live, and even trace down memory leaks.

Solving a Node.js Memory Leak on Heroku

```yaml
---
slug: use-github-as-a-cms
published: 2022-01-11T11:40:00-08:00
description: GitHub's rich GraphQL API gives you a built-in headless CMS
changelog: ~
---
```

First thing's first. I really love [Ghost](https://ghost.org/). It's got an excellent content and admin API and I'd honestly consider it better than many other Headless CMS options. Second, this isn't a post about Ghost. There were two major shortcomings in Ghost that ultimately I couldn't find good workarounds for.

1. Custom structured data. As it stands I was jamming everything into `codeinjection_foot` as YAML and hoping I didn't ever need that field. I also needed to comment my code explaining all my additional metadata resided in the field not called `metadata`. Yikes.
2. Something that isn't Mobiledoc. I understand why it's used. I don't like that my parsing process on Codedrift requires that I start from the final HTML and then throw it all in [rehype](https://github.com/jakobo/codedrift/blob/9d0bd286355f5b80d1ba71561d78603a9a388ed0/src/components/markup/rehype.js#L25) to [fix URL links, remove widowed text, and more](https://github.com/jakobo/codedrift/blob/0f5991456f576dd031f2e95888af5878a63f16c3/src/graphql/util.js).

HTML to AST back to HTML is not a good feeling and the pipeline was brittle. When I realized I didn’t want to re-add tags on my posts to Codedrift because of the complexity, I looked for alternatives.

I saw [Casual JavaScript](https://github.com/casualjavascript/blog) which was running the entire site off of [GitHub Issues](https://github.com/casualjavascript/blog/blob/gh-pages/index.html). It wasn't perfect, but GitHub Issues are in Markdown, and Markdown _does_ support frontmatter, and frontmatter _is literally designed for metadata_.

The missing piece was something that wasn't issue based. [Discussions](https://github.blog/changelog/2021-02-23-github-discussions-graphql-api-public-beta/). Sold.

## Core Tools
The original Codedrift had a GraphQL micro server that ran everything. It was overengineered (like all good personal projects should be), but in the rewrite I wanted to remove as much of my code as possible. The entire GraphQL infrastructure became a simple [http proxy](https://github.com/jakobo/codedrift/blob/9d5b0349dc204cd1076b54fd5e475dffe449c84d/src/pages/api/proxy/api.github.com/graphql.ts) using [http-proxy-middleware](https://www.npmjs.com/package/http-proxy-middleware). This way I wouldn't need to expose a personal access token in the browser.

The next step was to use [graphql-code-generator](https://www.graphql-code-generator.com/) for Typescript typed Urql hooks. I'm still a fan of Urql, and their [docs for next.js integration](https://formidable.com/open-source/urql/docs/advanced/server-side-rendering/#nextjs) are easy to follow and set up.

Because scope creep is a thing, the site's now also in Typescript. 🥳 

So now I've got typed GraphQL queries to GitHub's API without exposing the access token. All that's left is to get our Discussions, turn them into blog posts, and solve for weird cases like [tag-based searches](https://github.com/jakobo/codedrift/blob/main/src/pages/thunked/tag/%5Bname%5D.tsx#L19) which need to go through the `search` endpoint instead of the `repository` endpoint. Consistency (and sanity) between queries came from the Post fragment.

## The Post Fragment
Fragments are [better explained by graphql.org](https://graphql.org/learn/queries/#fragments), but in context they are the reusable unit of documents. In this case, a fragment on `Discussions` will let me get the same fields every time for our discussion, regardless if I arrived at the Discussion via the `search` endpoint or a `repository` endpoint. Even though it might overfetch a little for its use case, a fragment pushed through graphql-code-generator will give you a typed object representing the returned data structure.

In other words, having this GraphQL fragment
```gql
fragment PostDetails on Discussion {
  id
  title
  lastEditedAt
  url
  labels(first: 100) {
    nodes {
      id
      name
      description
    }
  }
  author {
    avatarUrl(size: 10)
    ... on User {
      id
      name
    }
  }
  body
}
```

Will give us this type in our generated file:
```ts
export type PostDetailsFragment = {
  __typename?: "Discussion";
  id: string;
  title: string;
  lastEditedAt?: any | null | undefined;
  url: any;
  body: string;
  labels?:
    | {
        __typename?: "LabelConnection";
        nodes?:
          | Array<
              | {
                  __typename?: "Label";
                  id: string;
                  name: string;
                  description?: string | null | undefined;
                }
              | null
              | undefined
            >
          | null
          | undefined;
      }
    | null
    | undefined;
  author?:
    | { __typename?: "Bot"; avatarUrl: any }
    | { __typename?: "EnterpriseUserAccount"; avatarUrl: any }
    | { __typename?: "Mannequin"; avatarUrl: any }
    | { __typename?: "Organization"; avatarUrl: any }
    | {
        __typename?: "User";
        id: string;
        name?: string | null | undefined;
        avatarUrl: any;
      }
    | null
    | undefined;
};
```

Even a `Partial<PostDetailsFragment>` adds excellent safeguards to a function when I expect to receive data shaped like the GraphQL response for a Discussion.

## Slugs and Search
Fetching the most recent posts is [a straightforward process](https://github.com/jakobo/codedrift/blob/cbcc0f4beac26e611c14cb789a7adc5b3ba7c55a/src/graphql/posts.graphql#L27) since the API can go from a repository to its discussions and limit those to a single category. For accessing a single post by anything other than its ID, there's only the `search` endpoint.

A generic search on GitHub takes a search query as a string. It's the same string used in [GitHub's Advanced Search](https://github.com/search/advanced?r=jakobo%2Fcodedrift) which at least means it's easy to test and iterate on, even if there is no official query builder. The result is a very generic GraphQL query for searching, which can be used for both slugs and tags. It could probably also be used for the recently created posts, but I'd prefer less magic strings where possible.

```gql
query SelectPostsWithSearch($search: String!, $first: Int = 1, $after: String) {
  search(first: $first, after: $after, type: DISCUSSION, query: $search) {
    nodes {
      ... on Discussion {
        ...PostDetails
      }
    }
    discussionCount
    pageInfo {
      endCursor
      hasNextPage
      hasPreviousPage
      startCursor
    }
  }
}
```

The first of two major downside to this approach is I need to constantly cast the node collection to `Discussion`, but I feel that's more my inexperience with Typescript than the language itself. The other is that because when running a search, there's always a risk of returning two records for a search. I briefly thought about changing the URLs to be `slug/post_id` but that felt like it was binding me too closely to GitHub's ID system.

## The New Post Workflow
Posts on Codedrift start in [my public notes](https://codedrift.com/notes), but eventually they’re well-formed enough to get ideas and opinions. At that point, there’s discussion on the draft, which used to exist exclusively over email and Twitter DMs. These processes also worked, but it always felt like I was missing an opportunity for more open discourse about a topic.

So now, when a post grows out of Codedrift, it’ll become an [issue](https://github.com/jakobo/codedrift/issues?q=is%3Aopen+is%3Aissue+label%3A%22%E2%9C%92+Draft%22), When it’s ready to be published, it can be moved to a Discussion, and GitHub saves all of the relevant comments and edits in the transfer. This has the added benefit of matching my mental model. Unposted content is an issue which certainly needs resolution. Posted content deserves a discussion.

The code for all of this is in the [Codedrift repository](https://github.com/jakobo/codedrift), though now that I'm just calling a GitHub API with no interference, the code feels very vanilla. And that feels pretty good.

GitHub's rich GraphQL API gives you a built-in headless CMS

fragment PostDetails on Discussion {
  id
  title
  lastEditedAt
  url
  labels(first: 100) {
    nodes {
      id
      name
      description
    }
  }
  author {
    avatarUrl(size: 10)
    ... on User {
      id
      name
    }
  }
  body
}


export type PostDetailsFragment = {
  __typename?: "Discussion";
  id: string;
  title: string;
  lastEditedAt?: any | null | undefined;
  url: any;
  body: string;
  labels?:
    | {
        __typename?: "LabelConnection";
        nodes?:
          | Array<
              | {
                  __typename?: "Label";
                  id: string;
                  name: string;
                  description?: string | null | undefined;
                }
              | null
              | undefined
            >
          | null
          | undefined;
      }
    | null
    | undefined;
  author?:
    | { __typename?: "Bot"; avatarUrl: any }
    | { __typename?: "EnterpriseUserAccount"; avatarUrl: any }
    | { __typename?: "Mannequin"; avatarUrl: any }
    | { __typename?: "Organization"; avatarUrl: any }
    | {
        __typename?: "User";
        id: string;
        name?: string | null | undefined;
        avatarUrl: any;
      }
    | null
    | undefined;
};


query SelectPostsWithSearch($search: String!, $first: Int = 1, $after: String) {
  search(first: $first, after: $after, type: DISCUSSION, query: $search) {
    nodes {
      ... on Discussion {
        ...PostDetails
      }
    }
    discussionCount
    pageInfo {
      endCursor
      hasNextPage
      hasPreviousPage
      startCursor
    }
  }
}


GitHub as a CMS

```yaml
---
slug: running-expo-in-windows-subsystem-for-linux-wsl2
published: 2021-11-10T18:00:00.000Z
description: Windows, meet Expo in Linux
repost:
  url: https://blog.expo.dev/running-expo-in-windows-subsystem-for-linux-wsl2-425f6fd7838e
  text: This article was written for the Expo Blog and appeared with the title _Running Expo in Windows Subsystem for Linux (WSL2)_. It's been reproduced here with permission.
changelog: ~
---
```

If it runs in Node.js, it can run Expo. Even better, if it runs in Linux, you can take advantage of all the excellent React Native tools that may not be utilizing cross-platform utilities like `cross-env`. Since Windows 10 has support for Linux through a virtualized Windows Subsystem for Linux (WSL2), we can use that to run our Expo applications, build tooling, and more.

Getting everything working on WSL2 **does** have a few gotchas, which is why this guide can help you get up and running quickly and avoid some of the most common pitfalls (spoiler: it’s networking related).

# Overview

- Setting up WSL2
- Configuring `WSLHostPatcher`
- Alternative: Using Windows Firewall Rules
- Final considerations

# Setting up WSL2 for Expo Development

WSL2 gives you access to a fully featured Linux command line inside of Windows. While great cross-platform tools like `env-cmd` exist, not every tool we use in React Native development is Powershell-aware. Using WSL2 eliminates those problems entirely; it’s just Linux. Unless specified, all our commands will be in WSL2’s Linux environment.

****What You’ll Need for This Step****

1. Windows 10 or 11 (with or without the Windows Insider Program)
2. Windows Subsystem for Linux ([Instructions here](https://docs.microsoft.com/en-us/windows/wsl/install))

The first thing we’re going to need is both Expo CLI and EAS CLI in our WSL environment. From there, we can initialize our standard Expo app, just like a managed app. We’re using the Typescript template, but you can also opt for a vanilla JS experience.

    $ npm install -g eas-cli
    $ npm install -g expo-cli
    $ expo init expo-cdc-example
    $ cd expo-cdc-example
    $ yarn start

We’ll get our Expo QR code, but when we try to visit our developer page, we’ll end up at a 404. This happens because to our **browser**, `localhost `is referring to Windows, and Windows doesn’t know that those Expo requests need to be forwarded along to the WSL.

# Configuring `WSLHostPatcher`

Unless you have specific security concerns (such as exposing your Windows machine to the Internet), the fastest way to fix the port forwarding problems is with a utility called `[WSLHostPatcher](https://github.com/CzBiX/WSLHostPatcher)`. Built on the Windows Detours framework, it dynamically updates the bind & listen addresses for your WSL instances. We’re going to download the utility and run it every time our WSL instances starts. (Adjust appropriately for a non-BASH environment)

    $ cd
    $ mkdir wsl; cd wsl
    $ wget https://github.com/CzBiX/WSLHostPatcher/releases/download/v0.1.0/release.zip
    $ unzip release.zip
    # run WSLHostPatcher on every login
    $ echo "./wsl/WSLHostPatcher.exe" >> ~/.bashrc

Let’s close our dev console, along with any running `yarn` instances, and reload WSL. We’ll be greeted with `WSLHostPatcher` confirming it found and updated our WSL instance.

    Dll path: \\wsl$\Ubuntu\home\<yourname>\wsl\WSLHostPatch.dll
    Found 1 WSL host
    Patched 1

Give `expo start` a try now, and watch how your `localhost` request automatically finds its way to your Expo instance.

# Alternative: Windows Firewall Setup

One of the downsides to `WSLHostPatcher` is that it will bind your WSL instance to `0.0.0.0`. For most development scenarios, this is perfectly ok. If you have security concerns though, you can also use a Powershell script to forward specific ports to your WSL instance. This is a much longer setup, but uses the built-in Windows Firewall.

First, you’ll want to download the following script based on the excellent work of [@edwindijas on github](https://github.com/microsoft/WSL/issues/4150#issuecomment-504209723), and save it to `c:\scripts\wslbridge.ps1`. This script does a couple things.

1. Removes all existing WSL2 based routing rules
2. Creates new Outbound & Inbound rules on the Windows machine for your specified ports
3. Removes any existing proxies between Windows and your WSL instance
4. Creates a proxy from Windows to your WSL instance for all specified ports

```ps1
# [Configuration]
# All the ports you want to forward separated by coma
$ports=@(7700,8088,8089,19000);

# You can change the addr to your ip
# config to listen to a specific address
$addr='0.0.0.0';

# [Init]
$remoteport = bash.exe -c "ifconfig eth0 | grep 'inet '"
$found = $remoteport -match '\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}';
if( $found ){
  $remoteport = $matches[0];
} else{
  echo "The Script Exited, the ip address of WSL 2 cannot be found";
  exit;
}

$ports_a = $ports -join ",";

# Remove Firewall Exception Rules
iex "Remove-NetFireWallRule -DisplayName 'WSL 2 Firewall Unlock' ";

# adding Exception Rules for inbound and outbound Rules
iex "New-NetFireWallRule -DisplayName 'WSL 2 Firewall Unlock' -Direction Outbound -LocalPort $ports_a -Action Allow -Protocol TCP";

iex "New-NetFireWallRule -DisplayName 'WSL 2 Firewall Unlock' -Direction Inbound -LocalPort $ports_a -Action Allow -Protocol TCP";

for( $i = 0; $i -lt $ports.length; $i++ ){
  $port = $ports[$i];
  iex "netsh interface portproxy delete v4tov4 listenport=$port listenaddress=$addr";
  iex "netsh interface portproxy add v4tov4 listenport=$port listenaddress=$addr connectport=$port connectaddress=$remoteport";
}
```

Open up ****Task Scheduler**** and set up a task that will run the script on login, after a small delay. For reference, your task should look like the following:

![Create Task dialog box](https://user-images.githubusercontent.com/1795/146844508-fbd9cb3e-57e5-41b2-8cfc-d38873d9662e.png)
_Creating a task in the Task Scheduler_

![Task Triggers tab](https://user-images.githubusercontent.com/1795/146844535-fa52152a-575e-440f-826b-c8e8bfd18662.png)
_Setting a Trigger condition for our task_

![The Actions tab](https://user-images.githubusercontent.com/1795/146844565-cd926b31-0d6a-4cd3-a010-976a16e3c8e8.png)
_Task Action set to start the `wslbridge.ps1` script with the `-ExecutionPolicy Bypass` flag, allowing it to run with the necessary permissions_

Once configured, either on login or by running the script manually, Windows will automatically forward ports to WSL for you. From here, you can now run `expo start` from within WSL, and `localhost` will resolve as expected.

# Other Things You Can Do With WSL And Expo

The fun doesn’t stop at just accessing Expo’s developer tools. Because you can forward any port to WSL, that means you can access the Chrome Debugger, use `adb`, integrate Flipper, monitor with Wireshark, the list goes on! You’ll also be running your Expo builds from within Linux, meaning any build tooling will work just as it would on a Linux or Mac workstation.

It’s a great time to build React Native apps, and now it’s easier than ever on Windows.


This article was written for the Expo Blog and appeared with the title _Running Expo in Windows Subsystem for Linux (WSL2)_. It's been reproduced here with permission.

# [Configuration]
# All the ports you want to forward separated by coma
$ports=@(7700,8088,8089,19000);

# You can change the addr to your ip
# config to listen to a specific address
$addr='0.0.0.0';

# [Init]
$remoteport = bash.exe -c "ifconfig eth0 | grep 'inet '"
$found = $remoteport -match '\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}';
if( $found ){
  $remoteport = $matches[0];
} else{
  echo "The Script Exited, the ip address of WSL 2 cannot be found";
  exit;
}

$ports_a = $ports -join ",";

# Remove Firewall Exception Rules
iex "Remove-NetFireWallRule -DisplayName 'WSL 2 Firewall Unlock' ";

# adding Exception Rules for inbound and outbound Rules
iex "New-NetFireWallRule -DisplayName 'WSL 2 Firewall Unlock' -Direction Outbound -LocalPort $ports_a -Action Allow -Protocol TCP";

iex "New-NetFireWallRule -DisplayName 'WSL 2 Firewall Unlock' -Direction Inbound -LocalPort $ports_a -Action Allow -Protocol TCP";

for( $i = 0; $i -lt $ports.length; $i++ ){
  $port = $ports[$i];
  iex "netsh interface portproxy delete v4tov4 listenport=$port listenaddress=$addr";
  iex "netsh interface portproxy add v4tov4 listenport=$port listenaddress=$addr connectport=$port connectaddress=$remoteport";
}


Running Expo in Windows Subsystem for Linux (WSL2)

```yaml
---
slug: relay-cursors-for-database-queries
published: 2021-06-07T00:00:00.000Z
description: You need more than a Primary Key
changelog: ~
---
```

[Cursors](https://relay.dev/graphql/connections.htm) are just strings, at least as far as GraphQL clients are concerned. It has absolutely no meaning beyond identifying a specific result in a set of edges. There’s no limits on its length; most are base64 encoded to emphasize that point. I'll come back to that in a moment..

Currently Code Drift is a Ghost shop, using their excellent [content API](https://ghost.org/docs/content-api/), which depends on `page` and `limit` for accessing and paging through records. However, Ghost’s `id` isn’t sufficient for implementing a cursor. The Ghost API doesn’t have a “all posts after `id:aedf0001...`” call, only access to those `page` and `limit` values. We could in theory get back to a given post in Ghost as long as we knew those values.

Which brings us back to the importance of cursors being opaque to a client. What if instead of using the content’s id, we also included the page and limit values that got us here?

    id:aedf001bcbc|page:2|limit:10

This works because while the client assumes the cursor to be opaque, the specification makes no such claims about the server. The server can (and should) care deeply about the cursor. In this example, our cursor can tell us how to reconstruct our query or API calls.

We can store our query’s state in our cursor. I promise, it isn’t as bad as it sounds.

## Metadata In the Cursor

By definition, cursors in Relay are a string-serializable value. While anything can go into the string, I’d recommend the [GraphQL Foundation’s advice](https://graphql.org/learn/pagination/#pagination-and-edges) of base64 encoding the value. For example, here’s a [query with a cursor](https://github.com/jakobo/codedrift/blob/c79a249/src/pages/index.js#L10) on Code Drift:

```gql
query {
  posts(first: 1) {
    edges {
      cursor
    }
  }
}
```

```json
{
  "data": {
    "posts": {
      "edges": [
        {
          "cursor": "eyJ0eXBlIjoiUG9zdCIsImZpbHRlciI6bnVsbCwib3JkZXJCeSI6InB1Ymxpc2hlZF9hdCBERVNDIiwiaWQiOiI2MDc3MzE1ZTU0OTBkNjAwM2IxM2FiNjEiLCJvZmZzZXQiOjB9"
        }
      ]
    }
  }
}
```

Our client shouldn’t care what’s in the cursor since it’s supposed to be opaque. But it’s **long** and starts with `ey` which usually means we’re going to see some JSON. Decoding the cursor shows us that it is just a JSON blob.

```js
new Buffer("eyJ0eXBlIjoiUG9zdCIsImZpb...", 'base64').toString();

/*
{
  "type": "Post",
  "filter": null,
  "orderBy": "published_at DESC",
  "id": "6077315e5490d6003b13ab61",
  "offset": 0
}
*/
```

It contains everything the server would need to reconstruct the original query including the `offset`. The next result is predictably `offset + 1`, assuming our query parameters such as filter and `orderBy` didn’t change.

## Encoding the ID

The second requirement for Relay is that our IDs need to be globally unique. For a small project like Code Drift, I trust Ghost to not double up my unique IDs. If I ever was concerned though, I'd probably just follow Hasura's model of encoding the ID as JSON arrays (which guarantees the order on serialization):

```js
const id = Buffer.from(
  JSON.stringify([
    1, // version id (for back-compat)
    "Post", // type of record
    ["aedf0..."], // keys array (uniquely ID this record)
  ])
).toString("base64");
```

If you do implement these custom IDs, you'll want to update any resolvers to spot these JSON encoded keys and unpack them.

## Reconstructing the Query

The biggest limitation of storing query reconstruction in the cursor is that you must revalidate the parameters. In our example, if the `orderBy` variable changed, then the `offset` is probably no longer correct. In most environments, you can probably just [add checks to ensure these parameters don’t fluctuate](https://github.com/Jakobo/codedrift/blob/5f74f80/src/graphql/resolvers/Query/posts.js#L26). In a complex case, you can implement custom business logic to “find” your result by its cursor and adjust your pagination accordingly.

Since most websites only deal with forward / backward pagination of the same result set (and therefore the same query parameters), you likely won’t need the complex case. It’s significantly more code.

It should also go without saying that your cursor should only have metadata in it. Don’t shove private values in there like an AWS key. Don’t make someone’s email address part of your cursor. In general, don’t put anything in there that you wouldn’t be willing to display on the client or expose in your GraphQL endpoints.

## Encoding Cursors on Return

The final step is to generate our special cursors. Every cursor needs the same data structure, allowing us to reconstruct the query regardless of which cursor the client sends along in the next request. The trickiest part is calculating the offset, but in most cases that will be `offset + N + 1` where `offset` was in your original cursor, and `N` is the Nth edge in your result set.

If it helps clean up the code, use a pair of `encodeCursor` and `decodeCursor` functions for consistency in your cursor payload. Because it’s base64 (and JSON) these cursors will be larger than they probably needed to be, but come with the added benefit of making it easier to reconstruct the query later.

This pattern is [used pretty heavily on Code Drift](https://github.com/jakobo/codedrift/blob/5f74f80/src/graphql/resolvers/Query/posts.js), specifically because every cursor must result in a RESTful query to Ghost’s content API. If Ghost ever implemented their own opaque cursors, I could switch over to those with a small GraphQL change. Until then though, this provides a crucial link between a Relay API and a traditional REST endpoint for an individual record regardless if it comes from an API, Firebase, or even a Postgres database.


query {
  posts(first: 1) {
    edges {
      cursor
    }
  }
}


new Buffer("eyJ0eXBlIjoiUG9zdCIsImZpb...", 'base64').toString();

/*
{
  "type": "Post",
  "filter": null,
  "orderBy": "published_at DESC",
  "id": "6077315e5490d6003b13ab61",
  "offset": 0
}
*/


const id = Buffer.from(
  JSON.stringify([
    1, // version id (for back-compat)
    "Post", // type of record
    ["aedf0..."], // keys array (uniquely ID this record)
  ])
).toString("base64");


Relay Cursors for Database Queries

```yaml
---
slug: complex-hasura-permissions-via-views
published: 2021-04-15T00:00:00.000Z
description: How you can solve complex Hasura permissions using postgres views
changelog: ~
---
```

As we continue to explore the space of mentorship & career success at [Aibex](https://aibex.com), one of our prototypes tackled the idea of focused group messaging; think Slack, but without the signing in to seventeen spaces just to get something done. To make a messaging app that focused on the needs of mentors and mentees required unique permissions. We wanted users to have access to any messages they were present for, even if they later ended up leaving the group. To use our Slack analogy, leaving a private channel shouldn’t cut you off from the information you were originally invited to see.

The prototype is currently running on [Hasura](https://hasura.io), generating our GraphQL API from an existing Postgres database. We chose Hasura to both reduce the amount of API code we needed to write and to take advantage of the systems’ in-built webhook actions. When it came time to implementing our permissions via Hasura’s JSON language, we ran afoul of Hasura’s undocumented Rules for Rules (my term, not theirs).

- Every Hasura user in the same "role" must also use the same rules
- You cannot use a computed column or (most) function calls in a rule
- The only dynamic variables you have at permission time is the Hasura session variables (such as `X-Hasura-User-Id`)

In a messaging app, any message could have one or more recipients. In the Aibex prototype specifically, we also wanted someone who joined a group chat to gain access to all prior messages in the chat. After a user left a chat, they could continue to see all the messages during their time in the group.

There is no amount of `or`, `and`, nor `_exists` that solves this by default. If the title didn't give it away, the solution lies in offloading all computational work to a [postgres view](https://hasura.io/docs/latest/graphql/core/databases/postgres/schema/views.html), simplifying the Hasura permission JSON.

## The Computed Conditional Problem

We're going to begin with a set of tables, defining a relationship between users, groups, and the messages within:

```sql
CREATE TABLE users (id uuid PRIMARY KEY);

CREATE TABLE groups (id uuid primary key);

CREATE TABLE memberships (
  id uuid PRIMARY KEY,
  user_id uuid REFERENCES users(id),
  group_id uuid REFERENCES groups(id),
  joined_at timestamp NOT NULL DEFAULT now(),
  departed_at timestamp
);

CREATE TABLE messages (
  id uuid PRIMARY KEY,
  from_id uuid REFERENCES users(id),
  group_id uuid REFERENCES groups(id),
  created_at timestamp NOT NULL DEFAULT now(),
  content text
);
```

![An Entity Relationship diagram showing a user belongs to many groups and that a group has many messages](https://user-images.githubusercontent.com/1795/146840050-5fc68478-b223-44ad-8c1b-b27e1aa17835.png)
_A user has many groups and that group has many messages_

The problem arises when we want to add our permissions to the `messages` table. Hasura's permission system allows us to access `(messages).group.memberships` via its inbuilt relationship system.

```json
{
  "group": {
    "memberships": {
      "user_id": {
        "_eq": "X-Hasura-User-Id"
      }
    }
  }
} 
```

Setting up our permissions this way gets us halfway; a user who has a relationship to the message (via the groups table), would be allowed to see the message in question. However, if we want to use `joined_at` and `departed_at`:

```json
{
  "group": {
    "memberships": {
      "_and": {
        "user_id": {
          "_eq": "X-Hasura-User-Id"
        },
        "departed_at": {
          "_lte": "<< CANNOT ACCESS messages.created_at >>"
        }
      }
    }
  }
}
```

Accessing the group through an Object Relationship will only let us test that the user was in the group at some point. By the time we've traversed the relationship to check the `departed_at` column, we've lost the ability to check the original row's `created_at` value. Variations that use `_exists` suffer from the same problem; currently you cannot use a row's dynamic value in permissions.

## Enter the View

To make our permission problem with Hasura, we need some **precomputed** version of our messages table. One that doesn't need `(messages).group.memberships.departed_at / created_at` to work. Even better, if we have one row for every recipient of a message, we can then just ask *that* version of the data instead.

One option would be to denormalize the data, creating copies of the message for every recipient, but doing so will create integrity problems down the line. Enter the view:

```sql
CREATE VIEW message_recipients AS
  SELECT msg.id, msg.group_id, mbr.user_id as recipient
  FROM (
    (messages msg JOIN groups grp ON (grp.id = msg.group_id))
      JOIN memberships mbr ON (mbr.group_id = grp.id)
  )
  WHERE (
    mbr.departed_at IS NULL OR msg.created_at <= mbr.departed_at
  );
```

The end result will be a view with a row for every message recipient, containing a message id and a user id, filtered such that the member is either active in the group (`NULL`) or their `departed_at` happened after the message's `created_at` timestamp. Now our Hasura permissions will work as expected. When we tell Hasura there is a **manual relationship** between `messages` and our new `message_recipients` view, it will link both objects together. We can then use the new relationship in our permission JSON.  🎉

```json
{
  "_or": {
    "from_id": {
      "_eq": "X-Hasura-User-Id"
    },
    "message_recipients": {
      "recipient": {
        "_eq": "X-Hasura-User-Id"
      }
    }
  }
}
```

## Where Views Go Wrong?

In practice, large datasets will make conditional permissions via views expensive. This is because a view is just a prior known subquery under the hood. There are optimizations you can do with heavy read data such as a [materialized view](https://www.postgresql.org/docs/9.3/rules-materializedviews.html), but those situations imply you’ve got both a substantial amount of traffic and a substantial amount of data. Before you throw all of this away though, take advantage of Hasura’s “Analyze” button. Missing an important index can have a huge performance cost and for most applications, views are perfectly ok.

And if you do outgrow the view, then you should consider denormalizing the data. Hasura Actions can take care of the denormalization for you. In our final example, there’s nothing that stops you from writing to a recipient table in response to new messages and users joining the group. For now though, you probably won’t need those optimizations.

Ultimately, views give Hasura’s rule engine superpowers. If you can write a query that selects for your permission, you can make a view. And if you can make a view, Hasura can understand it.

## Additional Reading

- [Hasura Row + Column Permission Tuples (GitHub)](https://github.com/hasura/graphql-engine/issues/3442)
- [Multiple Column + Row Permissions for the Same Role (Hasura Docs)](https://hasura.io/docs/latest/graphql/core/auth/authorization/role-multiple-rules.html)
- [Create View (Postgres Docs)](https://www.postgresql.org/docs/current/sql-createview.html)
- [Object and Array Relationships (Hasura Docs)](https://hasura.io/docs/latest/graphql/core/databases/postgres/queries/nested-object-queries.html)


How you can solve complex Hasura permissions using postgres views

CREATE TABLE users (id uuid PRIMARY KEY);

CREATE TABLE groups (id uuid primary key);

CREATE TABLE memberships (
  id uuid PRIMARY KEY,
  user_id uuid REFERENCES users(id),
  group_id uuid REFERENCES groups(id),
  joined_at timestamp NOT NULL DEFAULT now(),
  departed_at timestamp
);

CREATE TABLE messages (
  id uuid PRIMARY KEY,
  from_id uuid REFERENCES users(id),
  group_id uuid REFERENCES groups(id),
  created_at timestamp NOT NULL DEFAULT now(),
  content text
);


CREATE VIEW message_recipients AS
  SELECT msg.id, msg.group_id, mbr.user_id as recipient
  FROM (
    (messages msg JOIN groups grp ON (grp.id = msg.group_id))
      JOIN memberships mbr ON (mbr.group_id = grp.id)
  )
  WHERE (
    mbr.departed_at IS NULL OR msg.created_at <= mbr.departed_at
  );


Complex Hasura Permissions Via Views

```yaml
---
slug: coding-interviews
published: 2015-06-22T00:00:00.000Z
description: A rant on interviews that don't assess anything useful
changelog: ~
---
```

We shouldn't need to code in onsite interviews. However, in a sea of resumes, we need some kind of signal to understand how well a candidate understands the job they are applying for. Lacking any better way to answer "do you understand the concept of programming", we're left with technical interviews. We use these questions because we need to establish a baseline for programming competency.

As a company increases in size, our definition of competency changes. This is because as engineers, we are always told to [hire smart people](http://www.joelonsoftware.com/articles/GuerrillaInterviewing3.html). In response, our once straightforward baseline questions evolve to look for smarter and smarter developers. The cumulation of this process is a complete migration from "show me something you've built" to "reverse a linked list". As we try and find harder questions for candidates, we look online for guidance. We go to Google and we come back with knapsacks.

## Knapsacks in the Workplace

The Knapsack Problem illustrates everything that is wrong with technical interviews and how we conduct them today.

> Given a knapsack size of "n", what is the most valuable combination of items you can place in the knapsack without exceeding the weight limit?

![A knapsack showing a capacity of 15kg, surrounded by several generic blocks with various weights and prices](https://user-images.githubusercontent.com/1795/146829910-46154c35-282e-4724-85dd-490cfe2e0d52.png)
_Knapsack Problem Illustrated by Dake ([wikimedia commons](http://en.wikipedia.org/wiki/File:Knapsack.svg))_

Engineers love this shit.

- The basic implementation isn't "optimal"
- To do it "right" is "hard"
- The solutions on the internet are generally trusted

Hell, the Knapsack Problem even has one true solution and "hints" written by smart people that you can copy/paste right into your company wiki.

Everything about the Knapsack Problem is flawed. When we pick this question, we do it ignorant of why it was a "hard" problem to begin with.

The Knapsack Problem above is known as a Dynamic Programming (or DP) problem. DP was conceived in [1953 by Richard Bellman](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC528332/pdf/pnas00713-0067.pdf) as a way to break a large problem up into smaller sub problems. However, it wasn't until the late 1960's that [Viterbi](http://ieeexplore.ieee.org/xpl/login.jsp?tp=&amp;arnumber=1054010&amp;url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D1054010) as well as [Hart, Nilsson, and Raphael](http://ieeexplore.ieee.org/xpl/login.jsp?tp=&amp;arnumber=4082128&amp;url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D4082128) created algorithms that applied DP theory to real world problems. It took over 15 years to go from an idea to an algorithm, 15 years by some of the smartest people in our field.

**And we expect someone who doesn't recall the solution from memory to condense 15 years of computer science into a 45 minute window via clever hints.**

That's not just hard, that's unrealistic and unfair. The only way you could complete the programming activity in time is strict recall. This places the Knapsack Problem and its ilk in the realm of what I've been calling Eureka Problems; named for the required "A-Ha" someone has to have in the interview to stand a chance.

## Eureka Problems

The worst programming problems for technical interviews are in the bucket of Eureka Problems. If you don't know what a Eureka Problem feels like, here are a few technical interview questions.

1. Given an array of numbers, find the array range that contains the largest possible sum.
2. Given a grid and a robot at point [X1, Y1], write a solution that calculates the path to its destination [X2, Y2].
3. Given a linked list, write a solution that detects if the linked list contains a loop.

For these problems, it's possible to develop a solution to the problem. Anyone who has done a technical interview recently smells something missing: constraints. With **only a few more words,** these questions become Eureka Problems, containing one true answer.

1. Given an array of numbers, find the array range that contains the largest possible sum **in O(n) time**. (Solved using Kadane's algorithm)
2. Given a grid and a robot at point [X1, Y1], write a solution that calculates the **shortest** path to its destination [X2, Y2]. (Solved using A* search algorithm)
3. Given a linked list, write a solution that detects if the linked list contains a loop **without modifying our objects**? (Solved using Floyd's cycle-finding algorithm)

The restrictions force you to have a aptly named Eureka Moment for these problems. Like the Knapsack Problem above, you're asked to condense the thinking behind a decade or more of computer science into whatever time remains in the interview.

These Eureka Problems are so ingrained in our interview culture that there are [entire books written](http://www.amazon.com/Cracking-Coding-Interview-Programming-Questions/dp/098478280X) on coding interviews. That's right. Entire books center around recognizing the 10 most common Eureka Problems and the 10–15 variants of each. Those questions above are all in the book, along with the Knapsack Problem from higher up.

Once you add the restrictions, you are no longer assessing the candidate's ability to do the job asked of them. Instead, you've created a question that is assessing the candidate's memorization and recall ability. That's not what we want to know at all.

## Towards Pragmatism

"Do I want to be on the receiving end of this person's emails, code, & bug reports?"

This is the question you probably want to answer if you care about the software you are creating. It's also easier to check by asking questions closer to the job at hand. If he is doing browser code, make a multi-column layout. If she is an application developer, have her call a mock of your API and do something with the data.

We say we want a holistic developer who sees the big picture. Someone who asks questions about the business, how it all fits together, and sees opportunity in all the different parts of the role. None of these traits come out in asking someone to recreate a pathfinding algorithm.

Let's ask these instead, for example.

- Given this page on Stack Overflow (in an area your resume says you have experience in) can you help me figure out which answer is the right answer? How do we test it and find out?
- I have a design / product specification here. Can we walk through it and start on a basic proof of concept over the next 30 or so minutes?
- Let's take this code here that I wrote and isn't working. Can you sit over my shoulder as we debug this and help me create working code? We can also add some tests so I don't break it again.

If you can't get away from your attraction to classic Computer Science interview questions, consider accepting the less than perfect solution. You're hiring someone to solve problems and write code, not to design algorithms.
Remember.

"Do I want to be on the receiving end of this person's emails, code, & bug reports?" Because that's the question you need to be answering at the end of the interview.


A rant on interviews that don't assess anything useful