Sometimes APIs return data in a form that's not ideal for the client. We can deal with this by applying data transformation functions to the incoming data, to reshape it into exactly what we need.

A common use case is to render a list of items, and for each item, show info about some related item. For example, a list of comments might contain the author id, but visually we need to also show the author name.

// GET /comments
;[
  {
    id: 1,
    text: "Great post!",
    author_id: 1,
  },
  {
    id: 2,
    text: "I disagree!",
    author_id: 2,
  },
]

If you have the luxury of GraphQL, perhaps you can query for the list of comments, and the author name of each in a single request.

If you have a REST api which gives you separate endpoints for comments and authors, both must be fetched to get the complete picture.

// GET /authors
;[
  {
    id: "1",
    name: "Scott",
  },
  {
    id: "2",
    name: "Aron",
  },
]

In this case, when displaying comments, we'll need to look up the author for a given author_id.

Optimizing Arrays for Fast lookups

We could just store data in arrays. When we need to find an item, loop through the array to find the item with the matching unique identifier.

As the arrays grow larger, this becomes extremely inefficient, especially when nested within another loop:

// render list of comments

// loop through comment array
// for each comment ...
// get the comment's author by looping through author array
// gets slower as the number of authors grows

To make this more efficient, we could transform the arrays into maps, with ids for keys and items for values.

const authors = {
  1: {
    id: "1",
    name: "Scott",
  },
  2: {
    id: "2",
    name: "Aron",
  },
}

This makes look ups fast, but if we discard the array, we might lose important sort order information, so we can also store an array of keys, to keep that information in tact:

const authorKeys = ["1", "2"]

That way we can get the best of both worlds - a sorted array of items, and fast lookup of individual items by id. It's sort of like having a relational database table with a primary key index, except all in memory!

Map of Items, Array of Keys

A data structure we've found useful is a compound object that stores both a map of the items, keyed by id, and an array of the id keys:

const authors = {
  items: {
    1: {
      id: "1",
      name: "Scott",
    },
    2: {
      id: "2",
      name: "Aron",
    },
  },
  keys: ["1", "2"],
}

Transformation Function

Using JavaScript's handy reduce function, we can generate an efficient structure for lookups from the API response array of users with a single iteration through the array:

function parseItems(apiItems) {
  return apiItems.reduce(
    (acc, item) => {
      acc.items[item.id] = item
      acc.keys.push(item.id)
      return acc
    },
    {
      items: {},
      keys: [],
    },
  )
}

Now, when looping through comments, we can avoid iterating through the author array again and again for lookups!