Ginzoyel•2w ago

Bringing in 1-2 weeks of normal chat messages (banter)

Having this at our own URL. We'd like to support/display Discord channels that are not forums. Blacklisting them in robots.txt so these ones are not indexed but still show the community 'banter' from the past 1-2 weeks. Mostly as a way to show users that we have a fun and active community. We would not do this for many channels, but a few of those which have awesone discussion. We'd still use AO primarily for SEO. Is this at all possible, maybe with some changes to the bot? Please note that I would be more than happy to supply the feature myself via PR to Github. I just need to know what is already possible with the bot, as I assume now we catch only threads and not individual messages. Note that to avoid any rate-limiting or complaints on Discord's side, we'd only index a fair amount of regular channel messages/banter. Please see photo for an example of the banter/regular messages we want to display.

4 Replies

Rhys•2w ago

It is possible w/ the bot it's more a matter of displaying them, I can work something out here

GinzoyelOP•3d ago

I love you. You're going to get the fattest donation to Github 😄 our company is begging for this. Hey mate. Kust checking in, we'e fully moved over to AO now from Linen and my team keeps asking me when this will be done. This is a pretty high prio thing for us, is there something we can do to incentivise this? Can I help out in someway? I'm a full time backend developer.

Rhys•22h ago

Getting subpaths working for you now, will follow up on the making regular text channels displayed after Update here, have subpaths mostly working I see you've got an existing workaround for it that's a bit hacky, once this is ready it'll also do canonical urls for you and fix being able to access the host domain directly no action needed from you yet, once its ready i'll probably have you swap out your existing script Ok I've got subpaths working now with a proper setup The script you'll need to put in your Cloudflare worker is:

addEventListener("fetch", (event) => {
  event.respondWith(handleRequest(event.request));
});

async function handleRequest(request) {
  const SUBPATH = "/community"; 

  try {
    const urlObject = new URL(request.url);
    // If the request is to the subdirectory
    if (new RegExp(`^${SUBPATH}`).test(urlObject.pathname)) {
      // Then Proxy to Answer Overflow
      const DOCS_URL = "community.migaku.com";
      const CUSTOM_URL = "migaku.com";

      let url = new URL(request.url);
      url.hostname = DOCS_URL;

      // Strip SUBPATH from the pathname
      url.pathname = url.pathname.replace(new RegExp(`^${SUBPATH}`), "");
      // If the path becomes empty, set it to "/" to avoid issues with some servers
      if (url.pathname === "") {
        url.pathname = "/";
      }

      let proxyRequest = new Request(url, request);

      proxyRequest.headers.set("Host", DOCS_URL);
      proxyRequest.headers.set("X-Forwarded-Host", CUSTOM_URL);
      proxyRequest.headers.set("X-Forwarded-Proto", "https");
      proxyRequest.headers.set("X-AnswerOverflow-Skip-Subpath-Redirect", "True"); // Prevents being redirected to the main url
      return fetch(proxyRequest);
    }
  } catch (error) {
    // if no action found, play the regular request
    const newRequest = new Request(request);
    return await fetch(newRequest);
  }
  // If no action found and no error, play the regular request (with the added header)
  const newRequest = new Request(request);
  return await fetch(newRequest);
}

addEventListener("fetch", (event) => {
  event.respondWith(handleRequest(event.request));
});

async function handleRequest(request) {
  const SUBPATH = "/community"; 

  try {
    const urlObject = new URL(request.url);
    // If the request is to the subdirectory
    if (new RegExp(`^${SUBPATH}`).test(urlObject.pathname)) {
      // Then Proxy to Answer Overflow
      const DOCS_URL = "community.migaku.com";
      const CUSTOM_URL = "migaku.com";

      let url = new URL(request.url);
      url.hostname = DOCS_URL;

      // Strip SUBPATH from the pathname
      url.pathname = url.pathname.replace(new RegExp(`^${SUBPATH}`), "");
      // If the path becomes empty, set it to "/" to avoid issues with some servers
      if (url.pathname === "") {
        url.pathname = "/";
      }

      let proxyRequest = new Request(url, request);

      proxyRequest.headers.set("Host", DOCS_URL);
      proxyRequest.headers.set("X-Forwarded-Host", CUSTOM_URL);
      proxyRequest.headers.set("X-Forwarded-Proto", "https");
      proxyRequest.headers.set("X-AnswerOverflow-Skip-Subpath-Redirect", "True"); // Prevents being redirected to the main url
      return fetch(proxyRequest);
    }
  } catch (error) {
    // if no action found, play the regular request
    const newRequest = new Request(request);
    return await fetch(newRequest);
  }
  // If no action found and no error, play the regular request (with the added header)
  const newRequest = new Request(request);
  return await fetch(newRequest);
}

If you're fine with downtime you can put that in now and when I'm up I'll make the swap over, otherwise we can do it tomorrow with 0 / less downtime

Rhys•21h ago

Demo of it working over at https://rhys.ltd/idk While I do love the inguity of getting subpaths working with your workaround this sets up a few things more correctly, especially with the api & analytics

Search Engine Test V2

Search Engine Test V2 Community

View the Search Engine Test V2 Discord server on the web. Browse questions asked by the community and find answers.

Bringing in 1-2 weeks of normal chat messages (banter)

Did you find this page helpful?