michael/astro
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Add RSS generation (#123)

Browse source
This commit is contained in:
Drew Powers 2021-04-23 10:44:41 -06:00 committed by GitHub
parent 5eb232501f
commit 510e7920d2
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
21 changed files with 417 additions and 35 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

12
README.md
View file

@ -50,18 +50,18 @@ export default {
/** Set this to "preact" or "react" to determine what *.jsx files should load */ /** Set this to "preact" or "react" to determine what *.jsx files should load */
'.jsx': 'react', '.jsx': 'react',
}, },
/** Your public domain, e.g.: https://my-site.dev/ */
site: '',
/** Options specific to `astro build` */ /** Options specific to `astro build` */
buildOptions: { buildOptions: {
/** Your public domain, e.g.: https://my-site.dev/. Used to generate sitemaps and canonical URLs. */
site: '',
/** Generate sitemap (set to "false" to disable) */ /** Generate sitemap (set to "false" to disable) */
sitemap: true, sitemap: true,
}, },
/** Options for the development server run with `astro dev`. */ /** Options for the development server run with `astro dev`. */
devOptions: { devOptions: {
/** The port to run the dev server on. */ /** The port to run the dev server on. */
port: 3000 port: 3000,
} },
}; };
``` ```
@ -166,7 +166,7 @@ const localData = Astro.fetchContent('../post/*.md');
### 🗺️ Sitemap ### 🗺️ Sitemap
Astro will automatically create a `/sitemap.xml` for you for SEO! Be sure to set the `site` URL in your [Astro config][config] so the URLs can be generated properly. Astro will automatically create a `/sitemap.xml` for you for SEO! Be sure to set `buildOptions.site` in your [Astro config][config] so the URLs can be generated properly.
⚠️ Note that Astro won’t inject this into your HTML for you! You’ll have to add the tag yourself in your `<head>` on all pages that need it: ⚠️ Note that Astro won’t inject this into your HTML for you! You’ll have to add the tag yourself in your `<head>` on all pages that need it:
@ -181,7 +181,7 @@ Astro will automatically create a `/sitemap.xml` for you for SEO! Be sure to set
### 🍱 Collections (beta) ### 🍱 Collections (beta)
[Fetching data is easy in Astro](#-fetching-data). But what if you wanted to make a paginated blog? What if you wanted an easy way to sort data, or filter, say, by a given tag? When you need something a little more powerful than simple data fetching, Astro’s Collections API may be what you need. [Fetching data is easy in Astro](#-fetching-data). But what if you wanted to make a paginated blog? What if you wanted an easy way to sort data, or filter data based on part of the URL? Or generate an RSS 2.0 feed? When you need something a little more powerful than simple data fetching, Astro’s Collections API may be what you need.
👉 [**Collections API**][docs-collections] 👉 [**Collections API**][docs-collections]

47
docs/api.md
View file

@ -88,10 +88,57 @@ When using the [Collections API][docs-collections], `createCollection()` is an a
| `pageSize` | `number` | Specify number of items per page (default: `25`). | | `pageSize` | `number` | Specify number of items per page (default: `25`). |
| `routes` | `params[]` | **Required for URL Params.** Return an array of all possible URL `param` values in `{ name: value }` form. | | `routes` | `params[]` | **Required for URL Params.** Return an array of all possible URL `param` values in `{ name: value }` form. |
| `permalink` | `({ params }) => string` | **Required for URL Params.** Given a `param` object of `{ name: value }`, generate the final URL.\* | | `permalink` | `({ params }) => string` | **Required for URL Params.** Given a `param` object of `{ name: value }`, generate the final URL.\* |
| `rss` | [RSS][rss] | Optional: generate an RSS 2.0 feed from this collection ([docs][rss]). |
_\* Note: don’t create confusing URLs with `permalink`, e.g. rearranging params conditionally based on their values._ _\* Note: don’t create confusing URLs with `permalink`, e.g. rearranging params conditionally based on their values._
⚠️ `createCollection()` executes in its own isolated scope before page loads. Therefore you can’t reference anything from its parent scope. If you need to load data you may fetch or use async `import()`s within the function body for anything you need (that’s why it’s `async`—to give you this ability). If it wasn’t isolated, then `collection` would be undefined! Therefore, duplicating imports between `createCollection()` and your Astro component is OK. ⚠️ `createCollection()` executes in its own isolated scope before page loads. Therefore you can’t reference anything from its parent scope. If you need to load data you may fetch or use async `import()`s within the function body for anything you need (that’s why it’s `async`—to give you this ability). If it wasn’t isolated, then `collection` would be undefined! Therefore, duplicating imports between `createCollection()` and your Astro component is OK.
#### 📡 RSS Feed
You can optionally generate an RSS 2.0 feed from `createCollection()` by adding an `rss` option. Here are all the options:
```jsx
export async function createCollection() {
return {
async data({ params }) {
// load data
},
pageSize: 25,
rss: {
title: 'My RSS Feed',
description: 'Description of the feed',
/** (optional) add xmlns:* properties to root element */
xmlns: {
itunes: 'http://www.itunes.com/dtds/podcast-1.0.dtd',
content: 'http://purl.org/rss/1.0/modules/content/',
},
/** (optional) add arbitrary XML to <channel> */
customData: `<language>en-us</language>
<itunes:author>The Sunset Explorers</itunes:author>`,
/** Format each item from things returned in data() */
item: (item) => ({
title: item.title,
description: item.description,
pubDate: item.pubDate,
/** (optional) add arbitrary XML to each <item> */
customData: `<itunes:episodeType>${item.type}</itunes:episodeType>
<itunes:duration>${item.duration}</itunes:duration>
<itunes:explicit>${item.explicit || false}</itunes:explicit>`,
}),
},
};
}
```
Astro will generate an RSS 2.0 feed at `/feed/[collection].xml` (for example, `/astro/pages/$podcast.xml` would generate `/feed/podcast.xml`).
⚠️ Even though Astro will create the RSS feed for you, you’ll still need to add `<link>` tags manually in your `<head>` HTML:
```html
<link rel="alternate" type="application/rss+xml" title="My RSS Feed" href="/feed/podcast.xml" />
```
[config]: ../README.md#%EF%B8%8F-configuration [config]: ../README.md#%EF%B8%8F-configuration
[docs-collections]: ./collections.md [docs-collections]: ./collections.md
[rss]: #-rss-feed

3
docs/collections.md
View file

@ -2,7 +2,7 @@
## ❓ What are Collections? ## ❓ What are Collections?
[Fetching data is easy in Astro][docs-data]. But what if you wanted to make a paginated blog? What if you wanted an easy way to sort data, or filter, say, by a given tag? When you need something a little more powerful than simple data fetching, Astro’s Collections API may be what you need. [Fetching data is easy in Astro][docs-data]. But what if you wanted to make a paginated blog? What if you wanted an easy way to sort data, or filter data based on part of the URL? Or generate an RSS 2.0 feed? When you need something a little more powerful than simple data fetching, Astro’s Collections API may be what you need.
An Astro Collection is similar to the general concept of Collections in static site generators like Jekyll, Hugo, Eleventy, etc. It’s a general way to load an entire data set. But one big difference between Astro Collections and traditional static site generators is: **Astro lets you seamlessly blend remote API data and local files in a JAMstack-friendly way.** To see how, this guide will walk through a few examples. If you’d like, you can reference the [blog example project][example-blog] to see the finished code in context. An Astro Collection is similar to the general concept of Collections in static site generators like Jekyll, Hugo, Eleventy, etc. It’s a general way to load an entire data set. But one big difference between Astro Collections and traditional static site generators is: **Astro lets you seamlessly blend remote API data and local files in a JAMstack-friendly way.** To see how, this guide will walk through a few examples. If you’d like, you can reference the [blog example project][example-blog] to see the finished code in context.
@ -187,6 +187,7 @@ These are still paginated, too! But since there are other conditions applied, th
- [Fetching data in Astro][docs-data] - [Fetching data in Astro][docs-data]
- API Reference: [collection][collection-api] - API Reference: [collection][collection-api]
- API Reference: [createCollection()][create-collection-api] - API Reference: [createCollection()][create-collection-api]
- API Reference: [Creating an RSS feed][create-collection-api]
[docs-data]: ../README.md#-fetching-data [docs-data]: ../README.md#-fetching-data
[collection-api]: ./api.md#collection [collection-api]: ./api.md#collection

13
examples/blog/astro/pages/$posts.astro
View file

@ -18,7 +18,18 @@ export async function createCollection() {
allPosts.sort((a, b) => new Date(b.date) - new Date(a.date)); allPosts.sort((a, b) => new Date(b.date) - new Date(a.date));
return allPosts; return allPosts;
}, },
pageSize: 3 pageSize: 3,
rss: {
title: 'Muppet Blog',
description: 'An example blog on Astro',
customData: `<language>en-us</language>`,
item: (item) => ({
title: item.title,
description: item.description,
link: item.url,
pubDate: item.date,
}),
}
}; };
} }
--- ---

33
package-lock.json generated
View file

@ -1346,6 +1346,22 @@
"untildify": "^2.0.0" "untildify": "^2.0.0"
} }
}, },
"del": {
"version": "6.0.0",
"resolved": "https://registry.npmjs.org/del/-/del-6.0.0.tgz",
"integrity": "sha512-1shh9DQ23L16oXSZKB2JxpL7iMy2E0S9d517ptA1P8iw0alkPtQcrKH7ru31rYtKwF499HkTu+DRzq3TCKDFRQ==",
"dev": true,
"requires": {
"globby": "^11.0.1",
"graceful-fs": "^4.2.4",
"is-glob": "^4.0.1",
"is-path-cwd": "^2.2.0",
"is-path-inside": "^3.0.2",
"p-map": "^4.0.0",
"rimraf": "^3.0.2",
"slash": "^3.0.0"
}
},
"delayed-stream": { "delayed-stream": {
"version": "1.0.0", "version": "1.0.0",
"resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz", "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
@ -1849,6 +1865,11 @@
"integrity": "sha1-PYpcZog6FqMMqGQ+hR8Zuqd5eRc=", "integrity": "sha1-PYpcZog6FqMMqGQ+hR8Zuqd5eRc=",
"dev": true "dev": true
}, },
"fast-xml-parser": {
"version": "3.19.0",
"resolved": "https://registry.npmjs.org/fast-xml-parser/-/fast-xml-parser-3.19.0.tgz",
"integrity": "sha512-4pXwmBplsCPv8FOY1WRakF970TjNGnGnfbOnLqjlYvMiF1SR3yOHyxMR/YCXpPTOspNF5gwudqktIP4VsWkvBg=="
},
"fastq": { "fastq": {
"version": "1.11.0", "version": "1.11.0",
"resolved": "https://registry.npmjs.org/fastq/-/fastq-1.11.0.tgz", "resolved": "https://registry.npmjs.org/fastq/-/fastq-1.11.0.tgz",
@ -2473,6 +2494,18 @@
"resolved": "https://registry.npmjs.org/is-number/-/is-number-7.0.0.tgz", "resolved": "https://registry.npmjs.org/is-number/-/is-number-7.0.0.tgz",
"integrity": "sha512-41Cifkg6e8TylSpdtTpeLVMqvSBEVzTttHvERD741+pnZ8ANv0004MRL43QKPDlK9cGvNp6NZWZUBlbGXYxxng==" "integrity": "sha512-41Cifkg6e8TylSpdtTpeLVMqvSBEVzTttHvERD741+pnZ8ANv0004MRL43QKPDlK9cGvNp6NZWZUBlbGXYxxng=="
}, },
"is-path-cwd": {
"version": "2.2.0",
"resolved": "https://registry.npmjs.org/is-path-cwd/-/is-path-cwd-2.2.0.tgz",
"integrity": "sha512-w942bTcih8fdJPJmQHFzkS76NEP8Kzzvmw92cXsazb8intwLqPibPPdXf4ANdKV3rYMuuQYGIWtvz9JilB3NFQ==",
"dev": true
},
"is-path-inside": {
"version": "3.0.3",
"resolved": "https://registry.npmjs.org/is-path-inside/-/is-path-inside-3.0.3.tgz",
"integrity": "sha512-Fd4gABb+ycGAmKou8eMftCupSir5lRxqf4aD/vd0cD2qc4HL07OjCeuHMr8Ro4CoMaeCKDB0/ECBOVWjTwUvPQ==",
"dev": true
},
"is-plain-obj": { "is-plain-obj": {
"version": "2.1.0", "version": "2.1.0",
"resolved": "https://registry.npmjs.org/is-plain-obj/-/is-plain-obj-2.1.0.tgz", "resolved": "https://registry.npmjs.org/is-plain-obj/-/is-plain-obj-2.1.0.tgz",

2
package.json
View file

@ -46,6 +46,7 @@
"es-module-lexer": "^0.4.1", "es-module-lexer": "^0.4.1",
"esbuild": "^0.10.1", "esbuild": "^0.10.1",
"estree-walker": "^3.0.0", "estree-walker": "^3.0.0",
"fast-xml-parser": "^3.19.0",
"fdir": "^5.0.0", "fdir": "^5.0.0",
"find-up": "^5.0.0", "find-up": "^5.0.0",
"github-slugger": "^1.3.0", "github-slugger": "^1.3.0",
@ -94,6 +95,7 @@
"@typescript-eslint/eslint-plugin": "^4.18.0", "@typescript-eslint/eslint-plugin": "^4.18.0",
"@typescript-eslint/parser": "^4.18.0", "@typescript-eslint/parser": "^4.18.0",
"concurrently": "^6.0.0", "concurrently": "^6.0.0",
"del": "^6.0.0",
"domhandler": "^4.1.0", "domhandler": "^4.1.0",
"eslint": "^7.22.0", "eslint": "^7.22.0",
"eslint-config-prettier": "^8.1.0", "eslint-config-prettier": "^8.1.0",

39
src/@types/astro.ts
View file

@ -14,13 +14,16 @@ export interface AstroConfig {
astroRoot: URL; astroRoot: URL;
public: URL; public: URL;
extensions?: Record<string, ValidExtensionPlugins>; extensions?: Record<string, ValidExtensionPlugins>;
/** Public URL base (e.g. 'https://mysite.com'). Used in generating sitemaps and canonical URLs. */ /** Options specific to `astro build` */
site?: string;
/** Generate a sitemap? */
buildOptions: { buildOptions: {
/** Your public domain, e.g.: https://my-site.dev/. Used to generate sitemaps and canonical URLs. */
site?: string;
/** Generate sitemap (set to "false" to disable) */
sitemap: boolean; sitemap: boolean;
}; };
/** Options for the development server run with `astro dev`. */
devOptions: { devOptions: {
/** The port to run the dev server on. */
port: number; port: number;
projectRoot?: string; projectRoot?: string;
}; };
@ -34,7 +37,7 @@ export type AstroUserConfig = Omit<AstroConfig, 'buildOptions' | 'devOptions'> &
port?: number; port?: number;
projectRoot?: string; projectRoot?: string;
}; };
} };
export interface JsxItem { export interface JsxItem {
name: string; name: string;
@ -67,6 +70,34 @@ export interface CreateCollection<T = any> {
permalink?: ({ params }: { params: Params }) => string; permalink?: ({ params }: { params: Params }) => string;
/** page size */ /** page size */
pageSize?: number; pageSize?: number;
/** Generate RSS feed from data() */
rss?: CollectionRSS<T>;
}
export interface CollectionRSS<T = any> {
/** (required) Title of the RSS Feed */
title: string;
/** (required) Description of the RSS Feed */
description: string;
/** Specify arbitrary metadata on opening <xml> tag */
xmlns?: Record<string, string>;
/** Specify custom data in opening of file */
customData?: string;
/** Return data about each item */
item: (
item: T
) => {
/** (required) Title of item */
title: string;
/** (required) Link to item */
link: string;
/** Publication date of item */
pubDate?: Date;
/** Item description */
description?: string;
/** Append some other XML-valid data to this item */
customData?: string;
};
} }
export interface CollectionResult<T = any> { export interface CollectionResult<T = any> {

43
src/build.ts
View file

@ -10,8 +10,10 @@ import { fdir } from 'fdir';
import { defaultLogDestination, error, info } from './logger.js'; import { defaultLogDestination, error, info } from './logger.js';
import { createRuntime } from './runtime.js'; import { createRuntime } from './runtime.js';
import { bundle, collectDynamicImports } from './build/bundle.js'; import { bundle, collectDynamicImports } from './build/bundle.js';
import { generateRSS } from './build/rss.js';
import { generateSitemap } from './build/sitemap.js'; import { generateSitemap } from './build/sitemap.js';
import { collectStatics } from './build/static.js'; import { collectStatics } from './build/static.js';
import { canonicalURL } from './build/util.js';
const { mkdir, readFile, writeFile } = fsPromises; const { mkdir, readFile, writeFile } = fsPromises;
@ -20,12 +22,14 @@ interface PageBuildOptions {
dist: URL; dist: URL;
filepath: URL; filepath: URL;
runtime: AstroRuntime; runtime: AstroRuntime;
site?: string;
sitemap: boolean; sitemap: boolean;
statics: Set<string>; statics: Set<string>;
} }
interface PageResult { interface PageResult {
canonicalURLs: string[]; canonicalURLs: string[];
rss?: string;
statusCode: number; statusCode: number;
} }
@ -78,7 +82,7 @@ function getPageType(filepath: URL): 'collection' | 'static' {
} }
/** Build collection */ /** Build collection */
async function buildCollectionPage({ astroRoot, dist, filepath, runtime, statics }: PageBuildOptions): Promise<PageResult> { async function buildCollectionPage({ astroRoot, dist, filepath, runtime, site, statics }: PageBuildOptions): Promise<PageResult> {
const rel = path.relative(fileURLToPath(astroRoot) + '/pages', fileURLToPath(filepath)); // pages/index.astro const rel = path.relative(fileURLToPath(astroRoot) + '/pages', fileURLToPath(filepath)); // pages/index.astro
const pagePath = `/${rel.replace(/\$([^.]+)\.astro$/, '$1')}`; const pagePath = `/${rel.replace(/\$([^.]+)\.astro$/, '$1')}`;
const builtURLs = new Set<string>(); // !important: internal cache that prevents building the same URLs const builtURLs = new Set<string>(); // !important: internal cache that prevents building the same URLs
@ -97,12 +101,19 @@ async function buildCollectionPage({ astroRoot, dist, filepath, runtime, statics
} }
const result = (await loadCollection(pagePath)) as LoadResult; const result = (await loadCollection(pagePath)) as LoadResult;
if (result.statusCode >= 500) {
throw new Error((result as any).error);
}
if (result.statusCode === 200 && !result.collectionInfo) { if (result.statusCode === 200 && !result.collectionInfo) {
throw new Error(`[${rel}]: Collection page must export createCollection() function`); throw new Error(`[${rel}]: Collection page must export createCollection() function`);
} }
let rss: string | undefined;
// note: for pages that require params (/tag/:tag), we will get a 404 but will still get back collectionInfo that tell us what the URLs should be // note: for pages that require params (/tag/:tag), we will get a 404 but will still get back collectionInfo that tell us what the URLs should be
if (result.collectionInfo) { if (result.collectionInfo) {
// build subsequent pages
await Promise.all( await Promise.all(
[...result.collectionInfo.additionalURLs].map(async (url) => { [...result.collectionInfo.additionalURLs].map(async (url) => {
// for the top set of additional URLs, we render every new URL generated // for the top set of additional URLs, we render every new URL generated
@ -114,11 +125,17 @@ async function buildCollectionPage({ astroRoot, dist, filepath, runtime, statics
} }
}) })
); );
if (result.collectionInfo.rss) {
if (!site) throw new Error(`[${rel}] createCollection() tried to generate RSS but "buildOptions.site" missing in astro.config.mjs`);
rss = generateRSS({ ...(result.collectionInfo.rss as any), site }, rel.replace(/\$([^.]+)\.astro$/, '$1'));
}
} }
return { return {
canonicalURLs: [...builtURLs].filter((url) => !url.endsWith('/1')), // note: canonical URLs are controlled by the collection, so these are canonical (but exclude "/1" pages as those are duplicates of the index) canonicalURLs: [...builtURLs].filter((url) => !url.endsWith('/1')), // note: canonical URLs are controlled by the collection, so these are canonical (but exclude "/1" pages as those are duplicates of the index)
statusCode: result.statusCode, statusCode: result.statusCode,
rss,
}; };
} }
@ -186,10 +203,17 @@ export async function build(astroConfig: AstroConfig): Promise<0 | 1> {
const filepath = new URL(`file://${pathname}`); const filepath = new URL(`file://${pathname}`);
const pageType = getPageType(filepath); const pageType = getPageType(filepath);
const pageOptions: PageBuildOptions = { astroRoot, dist, filepath, runtime, sitemap: astroConfig.buildOptions.sitemap, statics }; const pageOptions: PageBuildOptions = { astroRoot, dist, filepath, runtime, site: astroConfig.buildOptions.site, sitemap: astroConfig.buildOptions.sitemap, statics };
if (pageType === 'collection') { if (pageType === 'collection') {
const { canonicalURLs } = await buildCollectionPage(pageOptions); const { canonicalURLs, rss } = await buildCollectionPage(pageOptions);
builtURLs.push(...canonicalURLs); builtURLs.push(...canonicalURLs);
if (rss) {
const basename = path
.relative(fileURLToPath(astroRoot) + '/pages', pathname)
.replace(/^\$/, '')
.replace(/\.astro$/, '');
await writeFilep(new URL(`file://${path.join(fileURLToPath(dist), 'feed', basename + '.xml')}`), rss, 'utf8');
}
} else { } else {
const { canonicalURLs } = await buildStaticPage(pageOptions); const { canonicalURLs } = await buildStaticPage(pageOptions);
builtURLs.push(...canonicalURLs); builtURLs.push(...canonicalURLs);
@ -239,18 +263,11 @@ export async function build(astroConfig: AstroConfig): Promise<0 | 1> {
} }
// build sitemap // build sitemap
if (astroConfig.buildOptions.sitemap && astroConfig.site) { if (astroConfig.buildOptions.sitemap && astroConfig.buildOptions.site) {
const sitemap = generateSitemap( const sitemap = generateSitemap(builtURLs.map((url) => ({ canonicalURL: canonicalURL(url, astroConfig.buildOptions.site) })));
builtURLs.map((url) => ({
canonicalURL: new URL(
path.extname(url) ? url : url.replace(/\/?$/, '/'), // add trailing slash if there’s no extension
astroConfig.site
).href,
}))
);
await writeFile(new URL('./sitemap.xml', dist), sitemap, 'utf8'); await writeFile(new URL('./sitemap.xml', dist), sitemap, 'utf8');
} else if (astroConfig.buildOptions.sitemap) { } else if (astroConfig.buildOptions.sitemap) {
info(logging, 'tip', `Set your "site" in astro.config.mjs to generate a sitemap.xml`); info(logging, 'tip', `Set "buildOptions.site" in astro.config.mjs to generate a sitemap.xml`);
} }
await runtime.shutdown(); await runtime.shutdown();

68
src/build/rss.ts Normal file
View file

@ -0,0 +1,68 @@
import type { CollectionRSS } from '../@types/astro';
import parser from 'fast-xml-parser';
import { canonicalURL } from './util.js';
/** Validates createCollection.rss */
export function validateRSS(rss: CollectionRSS, filename: string): void {
if (!rss.title) throw new Error(`[${filename}] rss.title required`);
if (!rss.description) throw new Error(`[${filename}] rss.description required`);
if (typeof rss.item !== 'function') throw new Error(`[${filename}] rss.item() function required`);
}
/** Generate RSS 2.0 feed */
export function generateRSS<T>(input: { data: T[]; site: string } & CollectionRSS<T>, filename: string): string {
let xml = `<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"`;
validateRSS(input as any, filename);
// xmlns
if (input.xmlns) {
for (const [k, v] of Object.entries(input.xmlns)) {
xml += ` xmlns:${k}="${v}"`;
}
}
xml += `>`;
xml += `<channel>`;
// title, description, customData
xml += `<title><![CDATA[${input.title}]]></title>`;
xml += `<description><![CDATA[${input.description}]]></description>`;
xml += `<link>${canonicalURL('/feed/' + filename + '.xml', input.site)}</link>`;
if (typeof input.customData === 'string') xml += input.customData;
// items
if (!Array.isArray(input.data) || !input.data.length) throw new Error(`[${filename}] data() returned no items. Can’t generate RSS feed.`);
for (const item of input.data) {
xml += `<item>`;
const result = input.item(item);
// validate
if (typeof result !== 'object') throw new Error(`[${filename}] rss.item() expected to return an object, returned ${typeof result}.`);
if (!result.title) throw new Error(`[${filename}] rss.item() returned object but required "title" is missing.`);
if (!result.link) throw new Error(`[${filename}] rss.item() returned object but required "link" is missing.`);
xml += `<title><![CDATA[${result.title}]]></title>`;
xml += `<link>${canonicalURL(result.link, input.site)}</link>`;
if (result.description) xml += `<description><![CDATA[${result.description}]]></description>`;
if (result.pubDate) {
// note: this should be a Date, but if user provided a string or number, we can work with that, too.
if (typeof result.pubDate === 'number' || typeof result.pubDate === 'string') {
result.pubDate = new Date(result.pubDate);
} else if (result.pubDate instanceof Date === false) {
throw new Error('[${filename}] rss.item().pubDate must be a Date');
}
xml += `<pubDate>${result.pubDate.toUTCString()}</pubDate>`;
}
if (typeof result.customData === 'string') xml += result.customData;
xml += `</item>`;
}
xml += `</channel></rss>`;
// validate user’s inputs to see if it’s valid XML
const isValid = parser.validate(xml);
if (isValid !== true) {
// If valid XML, isValid will be `true`. Otherwise, this will be an error object. Throw.
throw new Error(isValid as any);
}
return xml;
}

2
src/build/static.ts
View file

@ -10,7 +10,7 @@ export function collectStatics(html: string) {
const append = (el: Element, attr: string) => { const append = (el: Element, attr: string) => {
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion // eslint-disable-next-line @typescript-eslint/no-non-null-assertion
const value: string = $(el).attr(attr)!; const value: string = $(el).attr(attr)!;
if (value.startsWith('http')) { if (value.startsWith('http') || $(el).attr('rel') === 'alternate') {
return; return;
} }
statics.add(value); statics.add(value);

9
src/build/util.ts Normal file
View file

@ -0,0 +1,9 @@
import path from 'path';
/** Normalize URL to its canonical form */
export function canonicalURL(url: string, base?: string): string {
return new URL(
path.extname(url) ? url : url.replace(/(\/+)?$/, '/'), // add trailing slash if there’s no extension
base
).href;
}

35
src/runtime.ts
View file

@ -1,6 +1,6 @@
import { fileURLToPath } from 'url'; import { fileURLToPath } from 'url';
import type { SnowpackDevServer, ServerRuntime as SnowpackServerRuntime, SnowpackConfig } from 'snowpack'; import type { SnowpackDevServer, ServerRuntime as SnowpackServerRuntime, SnowpackConfig } from 'snowpack';
import type { AstroConfig, CollectionResult, CreateCollection, Params, RuntimeMode } from './@types/astro'; import type { AstroConfig, CollectionResult, CollectionRSS, CreateCollection, Params, RuntimeMode } from './@types/astro';
import type { LogOptions } from './logger'; import type { LogOptions } from './logger';
import type { CompileError } from './parser/utils/error.js'; import type { CompileError } from './parser/utils/error.js';
import { debug, info } from './logger.js'; import { debug, info } from './logger.js';
@ -26,7 +26,10 @@ interface RuntimeConfig {
} }
// info needed for collection generation // info needed for collection generation
type CollectionInfo = { additionalURLs: Set<string> }; interface CollectionInfo {
additionalURLs: Set<string>;
rss?: { data: any[] & CollectionRSS };
}
type LoadResultSuccess = { type LoadResultSuccess = {
statusCode: 200; statusCode: 200;
@ -78,6 +81,7 @@ async function load(config: RuntimeConfig, rawPathname: string | undefined): Pro
} }
const snowpackURL = searchResult.location.snowpackURL; const snowpackURL = searchResult.location.snowpackURL;
let rss: { data: any[] & CollectionRSS } = {} as any;
try { try {
const mod = await backendSnowpackRuntime.importModule(snowpackURL); const mod = await backendSnowpackRuntime.importModule(snowpackURL);
@ -90,11 +94,11 @@ async function load(config: RuntimeConfig, rawPathname: string | undefined): Pro
if (mod.exports.createCollection) { if (mod.exports.createCollection) {
const createCollection: CreateCollection = await mod.exports.createCollection(); const createCollection: CreateCollection = await mod.exports.createCollection();
for (const key of Object.keys(createCollection)) { for (const key of Object.keys(createCollection)) {
if (key !== 'data' && key !== 'routes' && key !== 'permalink' && key !== 'pageSize') { if (key !== 'data' && key !== 'routes' && key !== 'permalink' && key !== 'pageSize' && key !== 'rss') {
throw new Error(`[createCollection] unknown option: "${key}"`); throw new Error(`[createCollection] unknown option: "${key}"`);
} }
} }
let { data: loadData, routes, permalink, pageSize } = createCollection; let { data: loadData, routes, permalink, pageSize, rss: createRSS } = createCollection;
if (!pageSize) pageSize = 25; // can’t be 0 if (!pageSize) pageSize = 25; // can’t be 0
let currentParams: Params = {}; let currentParams: Params = {};
@ -116,6 +120,14 @@ async function load(config: RuntimeConfig, rawPathname: string | undefined): Pro
let data: any[] = await loadData({ params: currentParams }); let data: any[] = await loadData({ params: currentParams });
// handle RSS
if (createRSS) {
rss = {
...createRSS,
data: [...data] as any,
};
}
collection.start = 0; collection.start = 0;
collection.end = data.length - 1; collection.end = data.length - 1;
collection.total = data.length; collection.total = data.length;
@ -161,7 +173,10 @@ async function load(config: RuntimeConfig, rawPathname: string | undefined): Pro
return { return {
statusCode: 301, statusCode: 301,
location: reqPath + '/1', location: reqPath + '/1',
collectionInfo: additionalURLs.size ? { additionalURLs } : undefined, collectionInfo: {
additionalURLs,
rss: rss.data ? rss : undefined,
},
}; };
} }
@ -170,7 +185,10 @@ async function load(config: RuntimeConfig, rawPathname: string | undefined): Pro
return { return {
statusCode: 404, statusCode: 404,
error: new Error('Not Found'), error: new Error('Not Found'),
collectionInfo: additionalURLs.size ? { additionalURLs } : undefined, collectionInfo: {
additionalURLs,
rss: rss.data ? rss : undefined,
},
}; };
} }
@ -200,7 +218,10 @@ async function load(config: RuntimeConfig, rawPathname: string | undefined): Pro
statusCode: 200, statusCode: 200,
contentType: 'text/html; charset=utf-8', contentType: 'text/html; charset=utf-8',
contents: html, contents: html,
collectionInfo: additionalURLs.size ? { additionalURLs } : undefined, collectionInfo: {
additionalURLs,
rss: rss.data ? rss : undefined,
},
}; };
} catch (err) { } catch (err) {
if (err.code === 'parse-error') { if (err.code === 'parse-error') {

27
test/astro-rss.test.js Normal file
View file

@ -0,0 +1,27 @@
import { suite } from 'uvu';
import * as assert from 'uvu/assert';
import fs from 'fs';
import path from 'path';
import { execSync } from 'child_process';
import del from 'del';
import { fileURLToPath } from 'url';
const RSS = suite('RSS Generation');
const snapshot = `<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title><![CDATA[MF Doomcast]]></title><description><![CDATA[The podcast about the things you find on a picnic, or at a picnic table]]></description><link>https://mysite.dev/feed/episodes.xml</link><language>en-us</language><itunes:author>MF Doom</itunes:author><item><title><![CDATA[Rap Snitch Knishes (feat. Mr. Fantastik)]]></title><link>https://mysite.dev/episode/rap-snitch-knishes/</link><description><![CDATA[Complex named this song the “22nd funniest rap song of all time.”]]></description><pubDate>Tue, 16 Nov 2004 07:00:00 GMT</pubDate><itunes:episodeType>music</itunes:episodeType><itunes:duration>172</itunes:duration><itunes:explicit>true</itunes:explicit></item><item><title><![CDATA[Fazers]]></title><link>https://mysite.dev/episode/fazers/</link><description><![CDATA[Rhapsody ranked Take Me to Your Leader 17th on its list “Hip-Hop’s Best Albums of the Decade”]]></description><pubDate>Thu, 03 Jul 2003 06:00:00 GMT</pubDate><itunes:episodeType>music</itunes:episodeType><itunes:duration>197</itunes:duration><itunes:explicit>true</itunes:explicit></item><item><title><![CDATA[Rhymes Like Dimes (feat. Cucumber Slice)]]></title><link>https://mysite.dev/episode/rhymes-like-dimes/</link><description><![CDATA[Operation: Doomsday has been heralded as an underground classic that established MF Doom's rank within the underground hip-hop scene during the early to mid-2000s.
]]></description><pubDate>Tue, 19 Oct 1999 06:00:00 GMT</pubDate><itunes:episodeType>music</itunes:episodeType><itunes:duration>259</itunes:duration><itunes:explicit>true</itunes:explicit></item></channel></rss>`;
const cwd = new URL('./fixtures/astro-rss', import.meta.url);
const clear = () => del(path.join(fileURLToPath(cwd), '_site')); // clear _site output
RSS.before(() => clear());
RSS.after(() => clear());
RSS('Generates RSS correctly', async () => {
execSync('node ../../../astro.mjs build', { cwd: fileURLToPath(cwd) });
const rss = await fs.promises.readFile(path.join(fileURLToPath(cwd), '_site', 'feed', 'episodes.xml'), 'utf8');
assert.match(rss, snapshot);
});
RSS.run();

26
test/astro-sitemap.test.js Normal file
View file

@ -0,0 +1,26 @@
import { suite } from 'uvu';
import * as assert from 'uvu/assert';
import fs from 'fs';
import path from 'path';
import { execSync } from 'child_process';
import del from 'del';
import { fileURLToPath } from 'url';
const Sitemap = suite('Sitemap Generation');
const snapshot = `<?xml version="1.0" encoding="UTF-8"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"><url><loc>https://mysite.dev/episode/fazers/</loc></url><url><loc>https://mysite.dev/episode/rap-snitch-knishes/</loc></url><url><loc>https://mysite.dev/episode/rhymes-like-dimes/</loc></url><url><loc>https://mysite.dev/episodes/</loc></url></urlset>`;
const cwd = new URL('./fixtures/astro-rss', import.meta.url);
const clear = () => del(path.join(fileURLToPath(cwd), '_site')); // clear _site output
Sitemap.before(() => clear());
Sitemap.after(() => clear());
Sitemap('Generates Sitemap correctly', async () => {
execSync('node ../../../astro.mjs build', { cwd: fileURLToPath(cwd) });
const rss = await fs.promises.readFile(path.join(fileURLToPath(cwd), '_site', 'sitemap.xml'), 'utf8');
assert.match(rss, snapshot);
});
Sitemap.run();

4
test/fixtures/astro-dynamic/astro.config.mjs vendored
View file

@ -1,3 +1,5 @@
export default { export default {
sitemap: false, buildOptions: {
sitemap: false,
},
}; };

4
test/fixtures/astro-markdown/astro.config.mjs vendored
View file

@ -2,5 +2,7 @@ export default {
extensions: { extensions: {
'.jsx': 'preact', '.jsx': 'preact',
}, },
sitemap: false, buildOptions: {
sitemap: false,
},
}; };

5
test/fixtures/astro-rss/astro.config.mjs vendored Normal file
View file

@ -0,0 +1,5 @@
export default {
buildOptions: {
site: 'https://mysite.dev',
},
};

40
test/fixtures/astro-rss/astro/pages/$episodes.astro vendored Normal file
View file

@ -0,0 +1,40 @@
---
export let collection;
export async function createCollection() {
return {
async data() {
const episodes = Astro.fetchContent('./episode/*.md');
episodes.sort((a, b) => new Date(b.pubDate) - new Date(a.pubDate));
return episodes;
},
rss: {
title: 'MF Doomcast',
description: 'The podcast about the things you find on a picnic, or at a picnic table',
xmlns: {
itunes: 'http://www.itunes.com/dtds/podcast-1.0.dtd',
content: 'http://purl.org/rss/1.0/modules/content/',
},
customData: `<language>en-us</language>` +
`<itunes:author>MF Doom</itunes:author>`,
item: (item) => ({
title: item.title,
link: item.url,
description: item.description,
pubDate: item.pubDate,
customData: `<itunes:episodeType>${item.type}</itunes:episodeType>` +
`<itunes:duration>${item.duration}</itunes:duration>` +
`<itunes:explicit>${item.explicit || false}</itunes:explicit>`,
}),
}
}
}
---
<html>
<head>
<title>Podcast Episodes</title>
<link rel="alternate" type="application/rss+2.0" href="/feed/episodes.xml" />
</head>
<body></body>
</html>

13
test/fixtures/astro-rss/astro/pages/episode/fazers.md vendored Normal file
View file

@ -0,0 +1,13 @@
---
title: Fazers
artist: King Geedorah
type: music
duration: 197
pubDate: '2003-07-03 00:00:00'
description: Rhapsody ranked Take Me to Your Leader 17th on its list “Hip-Hop’s Best Albums of the Decade”
explicit: true
---
# Fazers
Rhapsody ranked Take Me to Your Leader 17th on its list “Hip-Hop’s Best Albums of the Decade”

13
test/fixtures/astro-rss/astro/pages/episode/rap-snitch-knishes.md vendored Normal file
View file

@ -0,0 +1,13 @@
---
title: Rap Snitch Knishes (feat. Mr. Fantastik)
artist: MF Doom
type: music
duration: 172
pubDate: '2004-11-16 00:00:00'
description: Complex named this song the “22nd funniest rap song of all time.”
explicit: true
---
# Rap Snitch Knishes (feat. Mr. Fantastik)
Complex named this song the “22nd funniest rap song of all time.”

14
test/fixtures/astro-rss/astro/pages/episode/rhymes-like-dimes.md vendored Normal file
View file

@ -0,0 +1,14 @@
---
title: Rhymes Like Dimes (feat. Cucumber Slice)
artist: MF Doom
type: music
duration: 259
pubDate: '1999-10-19 00:00:00'
description: |
Operation: Doomsday has been heralded as an underground classic that established MF Doom's rank within the underground hip-hop scene during the early to mid-2000s.
explicit: true
---
# Rhymes Like Dimes (feat. Cucumber Slice)
Operation: Doomsday has been heralded as an underground classic that established MF Doom's rank within the underground hip-hop scene during the early to mid-2000s.