Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Huge channels sometimes have slow search results #5

Closed
TheTechRobo opened this issue Jun 7, 2022 · 3 comments
Closed

Huge channels sometimes have slow search results #5

TheTechRobo opened this issue Jun 7, 2022 · 3 comments

Comments

@TheTechRobo
Copy link
Contributor

Not sure if this is just my Internet or the channel being huge, but figured I'd report it either way (if it's my Internet, you should add a way to increase the timeout).

https://discord.gg/2cujRs9K : in the #bot channel, scraping fails with

*** Task: ChannelDiscordTask (7 more)
Channel 514225574051446795 opened
Caught error while performing task: Error: Did not get search results after 10 seconds.
Saved screenshot to out/20220607T020524-resume/error.png
Stopping mitmdump
/home/thetechrobo/discard2/node_modules/brotli/build/encode.js:3
1<process.argv.length?process.argv[1].replace(/\\/g,"/"):"unknown-program");b.arguments=process.argv.slice(2);"undefined"!==typeof module&&(module.exports=b);process.on("uncaughtException",function(a){if(!(a instanceof y))throw a;});b.inspect=function(){return"[Emscripten Module object]"}}else if(x)b.print||(b.print=print),"undefined"!=typeof printErr&&(b.printErr=printErr),b.read="undefined"!=typeof read?read:function(){throw"no read() available (jsc?)";},b.readBinary=function(a){if("function"===
                                                                                                                                                                                                                              ^

Error: Did not get search results after 10 seconds.
    at performAndWaitForSearchResults (/home/thetechrobo/discard2/src/crawler/projects/discord/channel.ts:131:27)
    at async ChannelDiscordTask._searchAndClickFirstResult (/home/thetechrobo/discard2/src/crawler/projects/discord/channel.ts:136:29)
    at async ChannelDiscordTask.perform (/home/thetechrobo/discard2/src/crawler/projects/discord/channel.ts:253:24)
    at async Crawler.run (/home/thetechrobo/discard2/src/crawler/crawl.ts:258:28)
    at async crawler (/home/thetechrobo/discard2/src/cli.ts:71:5)
    at async Command.<anonymous> (/home/thetechrobo/discard2/src/cli.ts:148:9)

Here's the error screenshot:
error

@Sanqui
Copy link
Owner

Sanqui commented Jun 9, 2022

Wow, that search really takes >10 seconds. Fun...

@Sanqui Sanqui closed this as completed in 74a49a0 Jun 9, 2022
@exelotl
Copy link

exelotl commented Jun 17, 2022

Just posting to confirm that timeouts are causing issues for me too. My machine isn't quite fast enough to load the login page, as shown by the error screenshot:

error

@Sanqui
Copy link
Owner

Sanqui commented Jun 17, 2022

Interesting, I'm not quite sure that's a timeout issue given the "Not a well formed email address" error message, but a slow computer may play a role. I opened #10, thanks for the report!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants