llms.txt Endpoint

TinyKit Pro includes an llms.txt endpoint that provides structured site information optimized for Language Model consumption, following the llms.txt specification.

Overview

The /llms.txt endpoint dynamically generates a plain text file containing your site's essential information in a format that LLMs can easily parse and understand. This enhances AI integration and allows language models to better understand your application's purpose and structure.

Features

• 🤖 LLM-Optimized: Follows the llms.txt specification for maximum compatibility
• 🔄 Dynamic Generation: Content pulled from site settings in Convex database
• ⚡ Performance: Cached responses with 1-hour cache control
• 🛡️ Graceful Fallback: Default content if site settings unavailable
• 🎨 Customizable: Automatically reflects your branding and site configuration

Endpoint Details

URL

/llms.txt

Response Format

The endpoint returns plain text content in markdown format:

# [Your Site Name]

> [Your site description]

[Your site slogan]

Built with Next.js, Convex, and Stripe.

## Links

- [Home](https://yoursite.com): Main landing page
- [Sign Up](https://yoursite.com/auth/sign-up): Create an account
- [Sign In](https://yoursite.com/auth/sign-in): Access your account

HTTP Headers

Content-Type: text/plain; charset=utf-8
Cache-Control: public, max-age=3600, s-maxage=3600

Configuration

The llms.txt content is automatically generated from your site settings configured in the admin panel:

1. Navigate to Admin → Site Settings → Branding
2. Configure the following fields:
   • Site Name: Your application name
   • Slogan: Your site's tagline or motto
   • Description: Brief description of your application

Changes to these settings are immediately reflected in the /llms.txt endpoint after the cache expires (1 hour).

Implementation

Technical Details

File Location: src/app/llms.txt/route.ts

Key Features:

• Uses Next.js Route Handler with force-dynamic for real-time updates
• Fetches site settings via Convex fetchQuery
• Implements graceful fallback with generic content
• Optimized caching strategy (1 hour for successful responses, 1 minute for fallback)

Example Code:

export async function GET() {
  const baseUrl = process.env.NEXT_PUBLIC_SITE_URL ?? "http://localhost:3000";

  try {
    const settings = await fetchQuery(
      api.siteSettings.public.queries.getPublicSettings,
      {},
    );

    const content = `# ${settings.siteName}

> ${settings.description}

${settings.slogan ? `${settings.slogan}\n\n` : ""}Built with Next.js, Convex, and Stripe.

## Links

- [Home](${baseUrl}): Main landing page
- [Sign Up](${baseUrl}/auth/sign-up): Create an account
- [Sign In](${baseUrl}/auth/sign-in): Access your account
`;

    return new Response(content, {
      headers: {
        "Content-Type": "text/plain; charset=utf-8",
        "Cache-Control": "public, max-age=3600, s-maxage=3600",
      },
    });
  } catch {
    // Fallback content if settings unavailable
  }
}

Use Cases

AI Assistant Integration

Language models can use the llms.txt file to:

• Understand your application's purpose and features
• Provide accurate information about your site to users
• Generate appropriate responses when discussing your application
• Navigate and recommend relevant sections of your site

Search Engine Optimization

The llms.txt endpoint helps AI-powered search engines:

• Index your site more accurately
• Provide better search results
• Generate rich snippets for AI-powered search interfaces

Documentation for AI Tools

Development tools and AI coding assistants can:

• Understand your project structure
• Provide contextual help based on your application type
• Generate more relevant code suggestions

Best Practices

Optimizing Your llms.txt Content

1. Clear Site Name: Use your brand name or product name
2. Concise Description: Keep it under 200 characters for optimal parsing
3. Meaningful Slogan: Optional but helps convey your unique value proposition
4. Accurate Links: Ensure URLs are correct and accessible

Cache Considerations

• Production: 1-hour cache ensures good performance while allowing updates
• Development: Consider shorter cache times if testing frequently
• CDN Integration: The endpoint works seamlessly with CDN caching layers

Extending the llms.txt Endpoint

You can customize the endpoint to include additional information:

Adding More Sections

const content = `# ${siteName}

> ${description}

${slogan ? `${slogan}\n\n` : ""}Built with Next.js, Convex, and Stripe.

## Features

${features.map((f) => `- ${f.name}: ${f.description}`).join("\n")}

## Links

- [Home](${baseUrl}): Main landing page
- [Sign Up](${baseUrl}/auth/sign-up): Create an account
- [Sign In](${baseUrl}/auth/sign-in): Access your account
- [Documentation](${baseUrl}/docs): Learn more about our platform
`;

Dynamic Link Generation

Extend the endpoint to include dynamic links based on enabled features:

const links = [
  { url: `${baseUrl}`, label: "Home", description: "Main landing page" },
  {
    url: `${baseUrl}/auth/sign-up`,
    label: "Sign Up",
    description: "Create an account",
  },
  {
    url: `${baseUrl}/auth/sign-in`,
    label: "Sign In",
    description: "Access your account",
  },
];

// Add conditional links based on feature flags
if (settings.enableBlog) {
  links.push({
    url: `${baseUrl}/blog`,
    label: "Blog",
    description: "Latest updates",
  });
}

const linksSection = links
  .map((l) => `- [${l.label}](${l.url}): ${l.description}`)
  .join("\n");

Monitoring

Track llms.txt endpoint usage through:

• Server Logs: Monitor requests in Next.js logs
• Analytics: Track page views to /llms.txt endpoint
• Cache Performance: Monitor cache hit rates via CDN analytics

Troubleshooting

Endpoint Returns Fallback Content

Issue: The endpoint shows generic content instead of your custom settings.

Solutions:

1. Verify site settings are configured in admin panel
2. Check Convex deployment status
3. Ensure NEXT_PUBLIC_SITE_URL environment variable is set correctly (Note: this was previously SITE_URL in earlier versions)
4. Review server logs for Convex query errors

Content Not Updating

Issue: Changes to site settings not reflected in llms.txt.

Solutions:

1. Wait for cache expiration (1 hour)
2. Clear CDN cache if using a CDN
3. Verify site settings were saved successfully
4. Check browser cache and try in incognito mode

Invalid Format for LLMs

Issue: Language models not parsing content correctly.

Solutions:

1. Ensure content follows markdown format
2. Verify links are properly formatted
3. Check for special characters that might break parsing
4. Review llms.txt specification compliance

Related Documentation

• Site Branding - Configure site name, slogan, and description
• Architecture - Overall application architecture
• API Reference - Complete API documentation

← Back to Features | Next: Site Branding →