Bringing in 1-2 weeks of normal chat messages (banter)
Having this at our own URL. We'd like to support/display Discord channels that are not forums. Blacklisting them in
robots.txt
so these ones are not indexed but still show the community 'banter' from the past 1-2 weeks. Mostly as a way to show users that we have a fun and active community. We would not do this for many channels, but a few of those which have awesone discussion. We'd still use AO primarily for SEO.
Is this at all possible, maybe with some changes to the bot? Please note that I would be more than happy to supply the feature myself via PR to Github. I just need to know what is already possible with the bot, as I assume now we catch only threads and not individual messages.
Note that to avoid any rate-limiting or complaints on Discord's side, we'd only index a fair amount of regular channel messages/banter. Please see photo for an example of the banter/regular messages we want to display.
67 Replies
It is possible w/ the bot it's more a matter of displaying them, I can work something out here
I love you. You're going to get the fattest donation to Github 😄 our company is begging for this.
Hey mate. Kust checking in, we'e fully moved over to AO now from Linen and my team keeps asking me when this will be done. This is a pretty high prio thing for us, is there something we can do to incentivise this? Can I help out in someway? I'm a full time backend developer.
Getting subpaths working for you now, will follow up on the making regular text channels displayed after
Update here, have subpaths mostly working
I see you've got an existing workaround for it that's a bit hacky, once this is ready it'll also do canonical urls for you and fix being able to access the host domain directly
no action needed from you yet, once its ready i'll probably have you swap out your existing script
Ok I've got subpaths working now with a proper setup
The script you'll need to put in your Cloudflare worker is:
If you're fine with downtime you can put that in now and when I'm up I'll make the swap over, otherwise we can do it tomorrow with 0 / less downtime
Demo of it working over at https://rhys.ltd/idk
While I do love the inguity of getting subpaths working with your workaround this sets up a few things more correctly, especially with the api & analytics
Search Engine Test V2
Search Engine Test V2 Community
View the Search Engine Test V2 Discord server on the web. Browse questions asked by the community and find answers.
Perfect! So in AO do we still use community.oursite.com and then a Cloudflare Worker or how does it work?
I tried oursite.com/community in AOs' DNS but nothing happened. It seems only a subdomain is supported.
We were already using a Cloudflare Worker in the meantime but the metadata for everything is messed up and we don't want to waste our crawl budget.
Keep the community domain set how it is, I'll update it on my end


Yeah the metadata should be fixed with this approach
On the left is the results from the Cloudflare Worker (no metadata, bad thread names etc)
Alright I'll jump on this immediately.
Let me know when you have this set on your worker and I'll update it on my end as well
Will do.
Done! Please update it on your end 🙏
and thanks A LOT!
As promised you now have the monthly sponsor 😄
Updating now
Cool looks to be working, :ty: for the sponsor also!
Will polish it over the next few days for anything that pops up, also will get started on displaying regular channels
Pleasure mate. Thank you so much. We can drop you a few for the remaining features. You have no idea how much we appreciate this.
@Rhys Not sure if this is premature but I don't seem to be able to login to our Answer Overflow account.
Ah yeah this is on the migaku.com/community page?
Yep
That’s expected, I need to hide the button
Login in on those pages doesn’t really do anything at the moment - it will in the future but atm it’s fine to just hide no features will be lost
Ok no problem. Looks to be working otherwise.
Will be back in a few hours and will fix it then
Only thing I can see now is that in place of obfuscated user images, the page renders a tiny copy of the entire root domain i.e. https://oursite.com
We solved this previously, here was our worker script: https://pastebin.com/mbRYMXGh
Pastebin
CF Worker - Pastebin.com
Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

Thanks for also removing user page links for obfuscated users!
will fix!
@Ginzoyel did you change anything on the worker script? The https://migaku.com/community page isn't resolving to answer overflow
404
This content is not available on this domain.
Trying to figure out if it's my end or yours, looking on my end now
We're just using the Worker Script you posted before. Nothing else. It was working a few minutes ago 😮
Nothing else changed on our end
gotcha, taking a look at what might have changed on my end
So we've got nothing set up at the moment except the Worker Script you gave. All the other hacky stuff and the previous one I sent you on Pastebin we ditched. Just shared the Pastebin one so you could see how we handled obfuscated user images as those weren't resolving.
I'll check our end in the meantime but I'm 99% sure we've touched nothing since swapping out the Worker Script.
:ty: let me know if you see any errors i.e things being blocked in the workers script, could be requests are getting blocked somehow although it doesnt look like it
Sorry about the rough edges first time doing subpaths, once we get these worked out it should be smooth sailing 🙇♂️
I think I know what's happening :loading:
I think Cloudflare is blocking your requests, in your DNS settings for https://community.migaku.com/ turn off the Cloudflare proxy
404
This content is not available on this domain.

Done:

That shouldn’t be an A record, it should be a CNAME record set to cname.answeroverflow.com
Ok give me a moment.

This okay?
Yeah that looks right
Done
Still getting a 404.
The block that I was seeing is gone so that's good, although yeah it's still 404ing
I guess I don't need to touch anything on the AO dashboard?
No nothing there would impact it
I didn't change anything on my end also from when it was working, so I'm not quite sure where the problem is
https://www.whatsmydns.net/#CNAME/community.migaku.com
Looks good on this end.
DNS Propagation Checker - Global DNS Checker Tool
Instant DNS Propagation Check. Global DNS Propagation Checker - Check DNS records around the world.
I don't see requests coming in when I load the /community route and loading the subdomain with the bypass header loads the content
Can you add a header to the response from the Cloudflare worker?
We ultimately want to hide community.migaku.com entirely from Google Search, using a 301 Redirect or something else.
and use migaku.com/community
Sure, anything in particular?
Basically to this add some indicator
Lets do x-proxy-status-code and then the result code of the fetch call
Yeah it's setup to do that, if you go to community.migaku.com you'll see you get redirected to migaku.com/community with a 307 redirect - I can change that to 308 once we've got everything fully setup
That's great. I'll get the header added in, hang on.
Ok here is the new worker.
Do I still need to keep this

for the new worker to do its job?
Uh, not sure what that one is
Is that the old one? Maybe that’s the one that’s swallowing the routes
I'll delete it. I must have missed it. We needed a ton of stuff for it to work before and my boss wanted it now now now 😄
Sorry about the delay 😔 have been very busy with work
Hey seriously mate no problem you are saving our bacon here. We're a startup and dunno if you have worked for one but they always want everything 'yesterday' 😄
I did a test deployment to https://rhys.ltd/community and it works on that one so my gut feeling is it’s something on your end - still investigating to see though
Yeah I saw that one. I think it's our Cloudflare. I thought I deleted everything, I'll dump this one as well.
Only other thing I saw with yours yeah was that the images for obfuscated names didn't work when you enabled them.
Ah okay sorry that was an old tab of mine, I mean this:

is gone. But our worker isn't being triggered by anything. I was using that^ to trigger the worker before.
How are you triggering yours?
I added the domain to the worker
this might work for triggering it though as well

Got it like this now and it's already working again.
Just the issue with obfuscated username images that I can see off the top of my head.
Sounds good, I can fix that one now was just holding off on changes until the big one was fixed
So yeah in short, we had old proxy blocking everything and then nothing to trigger the new worker.
fixed
Champion
Would be great if we could have that 308 Permanent Redirect asap, our Google guy is going to try and get everything removed which is from community.migaku.com using the removal tool.
done
Looks good on our end! WOOT!
It seems there aren't any messages rolling in to our AO server after 3.6.2025? We've had a few posts since then. None are showing up.
@Rhys sorry to bother you mate, but could you do a quick resync or something? No idea why it has stopped syncing for us. We've got nothing newer than 03.06.2025 syncing anymore.
I signed in again to get the bot authorised but that didn't seem to trigger anything even after some hours.
Don't mean to distract you from working on the text channel/banter support. Hope it's not too difficult to force a refresh.
syncing should just work, don't kick the bot / change permissions as that might make things worse / break things
i've manually set it to first in the queue and started that to rerun, will check back in