mappings working for `gatsby develop` but failing for `gatsby build`

[travi]

Description

my goal is to set up relationships between details in frontmatter of several different groups of markdown files. i was able to get this working for gatsby develop with the approach to mapping that i found here. however, when i run gatsby build, it fails with a WebpackError due to failing to read a property of null, which is one of the mapped relationships that worked under gatsby develop. i was able to continue setting up more mappings that worked with gatsby develop but failed the same way with gatsby build.

Steps to reproduce

1. clone https://github.com/dsmjs/site
2. checkout the data-structure branch
3. nvm install
4. npm install
5. npm start (runs gatsby develop). load the (early WIP, so no judging :) ) site in the browser and note that things are rendering w/o issue
6. npm run build (runs gatsby build). note the webpack error

Expected result

gatsby build should complete successfully, resulting in a site that works the same as the one available from gatsby develop

Actual result

   8 |   return (
   9 |     <Meeting
> 10 |       sponsor={frontmatter.sponsor.frontmatter}
     |                                    ^
  11 |       meeting={frontmatter}
  12 |       host={frontmatter.host.frontmatter}
  13 |       talk={frontmatter.talk.frontmatter}


  WebpackError: Cannot read property 'frontmatter' of null

Environment

  System:
    OS: macOS High Sierra 10.13.5
    CPU: x64 Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
    Shell: 4.4.12 - /usr/local/bin/bash
  Binaries:
    Node: 9.4.0 - ~/development/travi/dotfiles/nvm/.nvm/versions/node/v9.4.0/bin/node
    npm: 5.6.0 - ~/development/travi/dotfiles/nvm/.nvm/versions/node/v9.4.0/bin/npm
  Browsers:
    Chrome: 67.0.3396.87
    Firefox: 50.0.1
    Safari: 11.1.1
  npmPackages:
    gatsby: 1.9.272 => 1.9.272 
    gatsby-link: 1.6.44 => 1.6.44 
    gatsby-plugin-glamor: 1.6.13 => 1.6.13 
    gatsby-plugin-google-fonts: 0.0.4 => 0.0.4 
    gatsby-plugin-react-helmet: 2.0.11 => 2.0.11 
    gatsby-remark-autolink-headers: 1.4.19 => 1.4.19 
    gatsby-remark-prismjs: 2.0.4 => 2.0.4 
    gatsby-source-filesystem: 1.5.39 => 1.5.39 
    gatsby-transformer-remark: 1.7.43 => 1.7.43 

File contents (if changed)

gatsby-config.js:

module.exports = {
  siteMetadata: {
    title: 'dsmJS  - Des Moines JavaScript User Group'
  },
  mapping: {
    'MarkdownRemark.frontmatter.talk': 'MarkdownRemark.frontmatter.title',
    'MarkdownRemark.frontmatter.speaker': 'MarkdownRemark.frontmatter.name',
    'MarkdownRemark.frontmatter.sponsor': 'MarkdownRemark.frontmatter.name',
    'MarkdownRemark.frontmatter.host': 'MarkdownRemark.frontmatter.location'
  },
  plugins: [
    {
      resolve: 'gatsby-transformer-remark',
      options: {
        plugins: [
          'gatsby-remark-prismjs',
          'gatsby-remark-autolink-headers'
        ]
      }
    },
    'gatsby-plugin-react-helmet',
    'gatsby-plugin-glamor',
    {
      resolve: 'gatsby-plugin-google-fonts',
      options: {
        fonts: [
          'quicksand:400,500,700'
        ]
      }
    },
    {
      resolve: 'gatsby-source-filesystem',
      options: {
        name: 'meetings',
        path: `${__dirname}/src/meetings`
      }
    },
    {
      resolve: 'gatsby-source-filesystem',
      options: {
        name: 'talks',
        path: `${__dirname}/src/talks`
      }
    },
    {
      resolve: 'gatsby-source-filesystem',
      options: {
        name: 'speakers',
        path: `${__dirname}/src/speakers`
      }
    },
    {
      resolve: 'gatsby-source-filesystem',
      options: {
        name: 'sponsors',
        path: `${__dirname}/src/sponsors`
      }
    },
    {
      resolve: 'gatsby-source-filesystem',
      options: {
        name: 'hosts',
        path: `${__dirname}/src/hosts`
      }
    }
  ]
};

package.json:

{
  "name": "site",
  "description": "Primary website for the dsmJS user-group",
  "license": "MIT",
  "author": "dsmJS",
  "maintainers": [
    "Matt Travi <npm@travi.org> (https://matt.travi.org/)"
  ],
  "private": true,
  "repository": "dsmjs/site",
  "bugs": "https://github.com/dsmjs/site/issues",
  "homepage": "https://github.com/dsmjs/site#readme",
  "scripts": {
    "lint:md": "globstar --node -- markdownlint **/*.md",
    "lint:js": "eslint . --cache",
    "test": "run-s lint:* build",
    "build": "gatsby build",
    "develop": "gatsby develop",
    "start": "run-s develop",
    "commitmsg": "commitlint -e",
    "precommit": "npm test",
    "deploy": "gh-pages -r \"https://$GH_TOKEN@github.com/dsmjs/site.git\" -d public/",
    "greenkeeper:update-lockfile": "greenkeeper-lockfile-update",
    "greenkeeper:upload-lockfile": "greenkeeper-lockfile-upload"
  },
  "config": {
    "commitizen": {
      "path": "./node_modules/cz-conventional-changelog"
    }
  },
  "devDependencies": {
    "@travi/eslint-config-travi": "1.6.14",
    "commitlint-config-dsmjs": "1.0.2",
    "cz-conventional-changelog": "2.1.0",
    "gh-pages": "1.2.0",
    "globstar": "1.0.0",
    "greenkeeper-lockfile": "1.15.1",
    "husky": "1.0.0-rc.9",
    "markdownlint-cli": "0.10.0",
    "npm-run-all": "4.1.3"
  },
  "dependencies": {
    "@dsmjs/components": "2.0.0-beta",
    "gatsby": "1.9.272",
    "gatsby-link": "1.6.44",
    "gatsby-plugin-glamor": "1.6.13",
    "gatsby-plugin-google-fonts": "0.0.4",
    "gatsby-plugin-react-helmet": "2.0.11",
    "gatsby-remark-autolink-headers": "1.4.19",
    "gatsby-remark-prismjs": "2.0.4",
    "gatsby-source-filesystem": "1.5.39",
    "gatsby-transformer-remark": "1.7.43",
    "prismjs": "1.15.0",
    "prop-types": "15.6.1",
    "react": "16.4.1",
    "react-dom": "16.4.1",
    "react-helmet": "5.2.0"
  }
}

gatsby-node.js:

const path = require('path');
const {createFilePath} = require('gatsby-source-filesystem');

exports.onCreateNode = ({node, getNode, boundActionCreators}) => {
  const {createNodeField} = boundActionCreators;

  if ('MarkdownRemark' === node.internal.type) {
    const slug = createFilePath({node, getNode, basePath: 'meetings'});
    createNodeField({
      node,
      name: 'slug',
      value: slug
    });
  }
};

exports.createPages = ({graphql, boundActionCreators}) => {
  const {createPage} = boundActionCreators;

  return graphql(`
      {
        allMarkdownRemark {
          edges {
            node {
              fields {
                slug
              }
            }
          }
        }
      }
    `).then(result => {
    result.data.allMarkdownRemark.edges.forEach(({node}) => {
      createPage({
        path: node.fields.slug,
        component: path.resolve('./src/templates/meeting.js'),
        context: {slug: node.fields.slug}
      });
    });
  });
};

gatsby-browser.js: N/A
gatsby-ssr.js: N/A

[pieh]

I would suspect this is hapenning in build only because some of your entries might not have linked sponsors (or there is a typo somewhere in markdown). This will happen only in build because we don't mount all pages in develop - this is done just client side when you open site in a browser.

Try adding some console logging before you try to access frontmatter.sponsor.frontmatter:

if (!frontmatter.sponsor) {
  console.log("this frontmatter doesn't have linked sponsor field", frontmatter)
}

you should see in console which entry is at fault here

[travi]

ah. your description made me realize what i missed without the debugging. the mappings were working fine for all actual meetings, but the slug creation was too aggressive now that markdown files existed for things other than meetings.

i added filtering so that slug creation that used the meeting template is now limited to meetings. i used a property that only exists on a meeting, which works but feels a bit hacky. since i'm still pretty new to gatsby and graphql, i'd be more than open to a suggestion if there is a more direct way to filter this.

either way, thanks a lot for helping me track down the issue and for your work on gatsby. i continue to be impressed the deeper i get.

[pieh]

👍 I'll close this, as issue was resolved

[travi]

yep that works. the only reason i didnt close myself was in case you had a recommendation for improving, but i didnt highlight the question very well:

i used a property that only exists on a meeting, which works but feels a bit hacky. since i'm still pretty new to gatsby and graphql, i'd be more than open to a suggestion if there is a more direct way to filter this.

what i have works to keep me moving and i can always improve as i learn, but if there is a recommended way, i'd love to learn tips like that early in my journey

[pieh]

Maybe you can check how it is done on gatsbyjs.org - https://github.com/gatsbyjs/gatsby/blob/master/www/gatsby-node.js#L278-L311

we get parent node (which should be File type) and that node would have sourceInstanceName property (which is what you set as name in gatsby-config.js), and use that to determine whether MarkdownRemark node is Meeting or Sponsor etc

[travi]

great, that sounds much more in line with what i was hoping for. i'll give that a shot for sure. thanks again!

[pauleveritt]

I have this working, albeit with painful behavior in develop mode. tl;dr Any edit loses the mapping for that node.

For Markdown, each post can map to one more more topics, where both are in Markdown. Let's say PostA has, in its front matter: topics: ["topicA", "topicB"] where those map to a value in the front matter of some markdown documents. sourceNodes setups up the mapping and makes the nodes, gatsby-config "mapping" does the mapping, the Topic.tsx works fine listing all those posts with that topic.

But if I edit any part of of post, the listing breaks. The query which grabs the topics fields, and extracts the front matter for the posts for the topic, returns all the front matter...except the one I just edited. Which is now null. Meaning, the mapping doesn't work for the edited target.

Restarting gatsby develop doesn't help. I have to blow away .cache and rebuild.

This makes for a suboptimal authoring experience.

I wondered if perhaps editing, from an immutability perspective, changed node.id but graphiql seems to have that be consistent.