build, doc: use new api doc tooling

[flakey5]

Switches over to using the new doc generation tooling. For more background on this, please see #52343

Currently a draft just to get feedback on the approach to this integration.

cc @nodejs/web-infra

---

Notable Change info (by @avivkeller):

The Node.js Website and Website Infrastructure teams have introduced a brand-new documentation pipeline, modernizing how our API docs are generated. While the documentation site may look familiar today, this infrastructure we are hard at work making a completely refreshed user interface in the very near future!

[nodejs-github-bot]

Review requested:

• @nodejs/nodejs-website
• @nodejs/web-infra

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.78%. Comparing base (1ad04e2) to head (3b3a9e2).
⚠️ Report is 75 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #57343      +/-   ##
==========================================
- Coverage   89.80%   89.78%   -0.03%     
==========================================
  Files         672      672              
  Lines      203907   203755     -152     
  Branches    39203    39167      -36     
==========================================
- Hits       183121   182940     -181     
- Misses      13113    13128      +15     
- Partials     7673     7687      +14     

see 38 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[ovflowd]

This is the result of many months of arduous work between many awesome folks, including @flakey5 @AugustinMauroy @araujogui @ovflowd @avivkeller and others.

I'm so proud of what we are achieving here and this is a huge step towards a modern tooling and a revamped API docs within Node.js

Approving, as I believe this is ready!

[lpinca]

RSLGTM because it is hard to review and outside of my comfort zone.

[aduh95]

• the additional keys (api, optional) (we can filter them out)

This isn't 1:1, sure, but it's certainly not a breaking change. A change that adds a new feature (or in this case, JSON key) shouldn't be considered breaking. The optional status of certain keys helps generate other output formats (i.e. the new generator creates a signature which relies on knowing whether a key is optional), and the api output is used internally to know where to output (${api}.json).

I disagree on the requirement that this change be reverted, it's not problematic (as I see it).

It's not about the change being breaking, it's about giving Node.js collaborators the opportunity to weigh in on it – like we do for any other change. It also makes it easier for backporting ro reverting if need be.

Does the consistency of the JSON order really matter? Per the JSON specification, "[a]n object is an unordered set of name/value pairs". Thus, the order shouldn't matter.

Tell that to V8, clearly it does matter otherwise you wouldn't have needed to add that .sort()

[ovflowd]

It also makes it easier for backporting ro reverting if need be.

With the fear of being pedantic here, what would really get backported here? There's nothing to be backported because these changes aren't breaking changes, both outputs from the old tooling and new tooling are completely compatible for anything consuming it.

Edit: What I meant here is that, there's no effort of actual backporting needed here as these changes are supposed to be backwards compatible. So these changes are safe to be backported without requiring outside/external or internal patches.

Tell that to V8, clearly it does matter otherwise you wouldn't have needed to add that .sort()

It matters because you structured you test to grab the keys in said order. That is an array, array are ordered. JSON objects aren't ordered. JavaScript objects "are", but we're talking about JSON here, not JavaScript.

[github-actions]

The notable-change PRs with changes that should be highlighted in changelogs. label has been added by @avivkeller.

Please suggest a text for the release notes if you'd like to include a more detailed summary, then proceed to update the PR description with the text or a link to the notable change suggested text comment. Otherwise, the commit will be placed in the Other Notable Changes section.

[avivkeller]

(I've added notable-change PRs with changes that should be highlighted in changelogs. , cause it'd be cool to highlight this effort)

[jasnell]

I've started working through the bits and pieces here and the conversation, working on trying to identify a compromise position that can allow this to proceed forward.

Here's where I'm landing currently but it might evolve a bit as I dig in further:

• Let's land this as a semver-major but modify it so that it is only used for new releases on current and new release lines. Existing release lines (e.g. 22.x, 24.x) would not initially use the updated format
• I agree with @avivkeller @ovflowd and @flakey5's assessments that the addition of the new fields should not be considered breaking... as acknowledged they are additive and are something that can be filtered out. I'm currently not seeing a justification for why those should be deferred to a separate PR
• I agree the JSON minification should be optional / behind a toggle

I get there are nuances in this that I probably don't yet fully have my head around so I'm continuing to catch up. Will comment further once I feel confident that I'm adequately up to speed.

[avivkeller]

@aduh95 if we land this as semver-major, will all your concerns be resolved? If so, could you label + approve the PR?

[aduh95]

I'm really hoping we don't end up landing this as semver-major – also there seems to be a consensus that this PR is very unlikely to be breaking. My preferred outcome would be that we land nodejs/doc-kit#595, include it in this PR, and the additional keys can be introduced later in follow up PRs.

[mcollina]

Sorry for the digression.

It matters because you structured you test to grab the keys in said order. That is an array, array are ordered. JSON objects aren't ordered. JavaScript objects "are", but we're talking about JSON here, not JavaScript.

Unfortunately this is imprecise. The order matters whenever it's processed by JavaScript.

Essentially:

{
  "foo": "bar",
  "foo": 42
}

It's a valid JSON document. When parsed in JS (and possibly everywhere), it would result into:

{
  foo: 42
}

Changing the order of keys mostly does not matter.

I don't think it matters in this case anyway, but please check for duplicates.

I don't think adding new properties is a breaking change either (removing would be). I would prefer them not be there unless strictly needed (as to reduce the surface area).

[ovflowd]

That's a good point. Although as you said correctly, doesn't apply here 😅

Edit: To be clear, I stand corrected by Matteo, he's technically correct and corrected my previous statement. I just don't think that in this case such issue could happen here.

[aduh95]

No it does apply, in the test file we have Object.values(json).at(-1)?.[0].introduced_in – because the JSON output has the structure it has, the key to access the file content varies depending on what's being documented (most of the times, it's modules, sometimes it's miscs, and once it's globals), the only invariant being that it's always the last key of the top-level object. If the keys get re-ordered and it's no longer the last key, it's a potential breaking change.
Why risking introducing a breaking change when it's so easy to re-order the keys when generating the JSON? I don't get it.

And FWIW I definitely agree that the current JSON output is in a less than ideal, and a reform would be welcome – but in a follow up semver-major PR.

[ovflowd]

I wanted to clarify a few points regarding the JSON structure and why the ordering has changed.

First, it's important to note that the top-level keys (which dictate the content order from the source Markdown) remain intact. We aren't introducing any breaking changes to the overall sequence of the document content.

Regarding the inner keys of each entry:

1. The ordering in the legacy tooling isn't actually semantic or "factually correct" based on the docs; it's simply an artifact of the order in which the legacy script tests for features.
2. In the new tooling, we handle these features in a different order simply because that's how the new logic was structured.

Since JSON objects are inherently unordered, this change shouldn't break any standard parsing. Unless a downstream consumer is accessing keys by index (e.g., Object.keys(json)[0]) rather than by name (e.g., json.type), the impact is purely visual.

While we can refactor the new tooling to mirror the legacy order if it’s considered a blocker, it feels like an unnecessary use of resources for a change that doesn't provide functional value. The current test failures seem to be a result of the tests expecting a specific string-matching order rather than validating the schema itself.

@aduh95, could you provide a real-world example of where this specific internal key order would break downstream usage? It would help to understand if there's a dependency I'm overlooking.

To clarify for everyone else, the concern is that keys like "type" appear higher up in the new output compared to the legacy output:

Note: The highlighted keys are just examples of the shift in placement. Even in the legacy tooling, this order wasn't always 100% consistent across different files.

[aduh95]

could you provide a real-world example of where this specific internal key order would break downstream usage?

Me trying to review this PR. With keys in a different order, it's very hard to review. With nodejs/doc-kit#595, at least I'm able to diff which allows me to see that e.g. node:process's 'message' event params are no longer parsed correctly:

The HTML version is also suffering from the lack of parsing:

I haven't investigated, that's likely due to the conflict on tools/doc/type-parser.mjs rather than an actual bug in doc-kit, hopefully though it does demonstrate how important it is for reviewing.

The sub-keys being in a different order does produce a lot of diff in the output, it's annoying but I guess it's workable. BTW there are indeed lots of bug fixes, so good work!

[avivkeller]

Me trying to review this PR. With keys in a different order, it's very hard to review. With nodejs/doc-kit#595, at least I'm able to diff which allows me to see that e.g. node:process's 'message' event params are no longer parsed correctly:

That's an issue with the markdown. See #61756

[ovflowd]

Me trying to review this PR. With keys in a different order, it's very hard to review. With nodejs/doc-kit#595, at least I'm able to diff which allows me to see that e.g. node:process's 'message' event params are no longer parsed correctly:

Gotcha, for manual comparison of the JSON this makes sense. I guess we can land nodejs/doc-kit#595

Me trying to review this PR. With keys in a different order, it's very hard to review. With nodejs/doc-kit#595, at least I'm able to diff which allows me to see that e.g. node:process's 'message' event params are no longer parsed correctly:

That's an issue with the markdown. See #61756

Do you mean to say the "node:process's 'message' event params are no longer parsed correctly:" is an issue on the markdown or?