This document discusses processes, procedures, and practices for the update of this CLDR site, https://cldr.unicode.org

General Information

Hosting

The site, https://cldr.unicode.org is automatically deployed from changes to the main branch of the docs/site directory of the CLDR repo. Hosting and build services are provided by CloudFlare, on a worker named cldr-next.

Proposing Changes

Changes to the site are effected by means of GitHub pull requests. When a PR is created that updates docs/site, a special label is automatically added, Site Update

If the branch is on the “head fork” (the CLDR upstream repository), then a preview link is generated for each commit.

This is the recommended practice, so that the site can be previewed exactly as it will be deployed. Create a branch with your GitHub username, such as srl295/CLDR-1234 This is different from how we normally create CLDR feature branches in a personal fork. PRs that are NOT on the head fork will be harder to review because we can’t preview them. Review such changes carefully, since merging to main updates the site.

The preview links are comments on the pull request, titled for example Deploying with Cloudflare Workers. There will be a commit preview URL, an a branch preview URL. The commit preview URL such as https://16221760-cldr-next.unicode.workers.dev/ shows what a specific commit looks like, whereas the branch preview URL such as https://srl295-site-how-to-update-cldr-18019-site-cldr-next.unicode.workers.dev/ is a link that remains valid even if that PR gets an additional update.

Hint: If you want to share a link to review the PR, you will usually share the branch preview URL so that the latest version of the PR can be reviewed.

When visiting a preview URL, click on the link in the right sidebar to visit the Sitemap (example link). In a preview deployment, there’s a special yellow box that appears at the top of the Sitemap indicating which site pages have changed in this PR, for example:

Redirects

See the _redirects file for documentation on the redirects process.

Authoring Site Content

We follow Github Flavored Markdown plus some extensions. The Github Guide can provide helpful guidance on the format.

File Naming

All Markdown filenames end in .md, but the .md is removed from the final URL.

For example:

Note that “development” is on both the name of a .md file, and a directory containing other files.

As a historical artifact, there are a number of pages as of this writing are under the /docs/site/index subdirectory, such as https://cldr.unicode.org/index/charts. There is an effort CLDR-18575 to drop /index/ as a subdirectory, leaving only index.md as the top level homepage and many other paths collapsed by one segment. The directory named /index has no special significance.

All files must be listed in the sitemap or the build will fail.

sitemap

• sitemap.tsv is a tsv (tab separated values) file that puts every file in the right location. That is, it controls what goes in the Subpages of each directory.
• Empty lines and lines beginning with # are ignored.
• Be very careful with the tabs, since they control the structure. The tab column (aka indentation) shows which directory each page is in. Do not replace tabs by spaces.
• Maintain the tabs in the document so that the file can be displayed on GitHub.
• The number of tabs at the beginning of each line determines the page’s depth in the sitemap. The single index line has 0 tabs, and every subsequent line has 1 or more tabs at the beginning of the line.
• Every .md file in site/, at every level, must be in site/sitemap.tsv. Conversely, every uncommented, non-empty line in site/sitemap.tsv must have a corresponding .md file. However, do not include the “.md” prefix in file names.
• NOTE: the directory structure in github determines URL for pages, but does not correspond to the sitemap structure. For example:

• The easiest way to edit is to copy the contents into a spreadsheet, edit, then copy back.
  • Remember to copy into a spreadsheet each time you make a branch, so that you get others’ changes to the sitemap.tsv.

Titles and Frontmatter

Add frontmatter (two lines with triple dashes ---) with a title at the top of the document. The title should be quoted (single or double quotes) if it contains any special characters. It’s always safe to quote.

---
title: "This is: My page title (New!)"
---

## This is the first heading

This is a paragraph

## This is the second heading

This is another paragraph

Note that the headings begin at ## (second level). This is because the first level (h1) heading is an older way to represent the title. If there is also an `h1 heading, it will be ignored in favor of the title. The renderer will automatically repeat the title inside the document.

---
title: "This is my title"
---

# ~This line is extraneous, remove it~

## First heading…

Links

Links within the cldr.unicode.org site

Links to other pages within the site must be host-relative links.

Example: to link to the heading Adding New Locales under the source file /docs/site/requesting_changes.md

Use this link: [Adding New Locales](/requesting_changes#adding-new-locales)

Note that it is a host-relative link, beginning with /.

Don’t use absolute links to the site or subpages of https://cldr.unicode.org (nor http://cldr.unicode.org) from within the site.

Also, don’t link to .md or .html files within the site, instead just link to the ‘base name’ of the file (without the .md or .html suffix).

NOTE: CLDR-18011 will fix the link format to be relative links to the .md files.

Links to Sections

A header such as ## Who uses CLDR? automatically acts as an anchor to #who-uses-cldr — there is no reason to add an explicit anchor.

Don’t change any header text without an additional change. If you really must rename a header, you can add an additional custom anchor to the old name so that our links are never broken (our policy). Use the following format whenever you do this, with the custom anchor on the line before the heading.

Example:

OLD:

### Old Heading

NEW:

<a name="old-heading"></a>
### New Heading

Techniques and Procedures

Testing Locally

See https://github.com/unicode-org/cldr/blob/main/tools/scripts/web/docker/README.md for instructions on locally testing the site.

Making changes using the GitHub.com web interface

You can complete the entire editing process using a browser logged in to github.com. Here are the steps:

(1) Open a ticket and locate your source file

In this example, hypothetical CLDR ticket “CLDR-98765” asks for a modification to the page: https://cldr.unicode.org/index/cldr-spec/core-data-for-new-locales

This corresponds to the markdown file: https://github.com/unicode-org/cldr/blob/main/docs/site/index/cldr-spec/core-data-for-new-locales.md

(2) Select the file to change

• Navigate to the markdown file that corresponds to the page being changed, in this example: https://github.com/unicode-org/cldr/blob/main/docs/site/index/cldr-spec/core-data-for-new-locales.md; that is easiest by searching for a prefix of the file name, eg core-data-for, then hitting enter/return.
• then click on the “edit” (pencil) icon

(3) Make the required edits to the file

Using the GitHub editor, you can switch between the “Edit” (raw markdown text) and “Preview” (a display of how the text will be rendered). This will be close to the final format, but not exactly the same.

(4) Commit the changes

• Click the green “Commit changes” button (above the editing window to the right).
• In the dialog, modify the commit message, since it must start with the CLDR ticket number, then a space (that is important), then by a brief summary of the changes in this commit.
• For the first file, you will be asked to create a branch. Give it a name matching the commit format for the ticket, e.g. CLDR-98765 Brief description… (Only one space between the number and the description. The description must begin with a letter from A-Z or a-z.)
• Click the green “Commit changes” button. This commit (and any others made to this branch) will become part of the same pull request.
• You will be asked to create a branch. Name the branch beginning with the ticket number, optionally including a brief description. Examples: CLDR-98765 or CLDR-98765/site-update-infohub

(5) Create the pull request

• Click on “Pull requests” at the top of the window. There should be a notification that the branch you created in step (1) had recent activity. In this example: CLDR-98765 had recent pushes
• Click on the green button “Compare & pull request”
• Complete the description section (by inserting the CLDR ticket number and possibly ticking the “completes ticket” box).
• Click on the green button “Create pull request”. Observe creation of pull request https://github.com/unicode-org/cldr/pull/xxxx

(6) Adding additional commits

If further changes are needed (perhaps in response to review feedback or because more than one file needs to be changed), you can add additional commits to this branch. These additional commits will become part of this pull request.

• Open the pull request.
• At the top you’ll see something like the following:
• Click on the from value, eg CLDR-17995-Fix…
• That puts you on the right branch.
• Repeat steps (2) through (4) for each additional file to be edited.

Note: If you accept review feedback suggestions in the pull request, make sure that you accept them with a properly formatted commit message, as simple as: CLDR-17995 accept review comments

(7) Never change page names

• We do not change the names of pages, except in very limited circumstances.
  1. We require older URLs to be stable, and never break.
  2. We have a sitemap that controls the structure of the site: which pages contain others.
  3. Changing a page name requires adding redirects for all old links, and requires fixing the sitemap.

(8) Adding a new page or directory

• As in #6, get to the right branch.
• Find the directory you want the new page in, and hit “Add File”
• 
• Type in the new file name (xxx.md for a file, xxx for a directory)
• After creating the file, modify the sitemap.tsv (see step 9)
• If it is a folder, then in that folder’s containing folder, create a .md file with the same name.
  • For example, for a directory site/development/foobar, also create a file site/development/foobar.md that describes the contents of that directory.
  • (Note: Don’t manually list the subpages of that directory in the .md file, because the Subpages sidebar shows them.)

(9) Modifying the site/sitemap.tsv

See sitemap for instructions.