A few updates here for you folks
Fixing the first requirement
Like what I thought the first requirement is quite simple. I use JSON pretty print to help me get a better and clean look at the returned JSON when testing. But not to mention this is my first time actually running the docker and start all the images by using npm services:start and every time you made some change to the code you will have to stop and run it again npm services:stop.
I hear that Emily has been working on the invalid feeds for a long time so I thinking about checking her commit would give me an idea on where to start it. I see Emily create some change in the src/api/posts/src/storage.js where we might get the info of the invalid feed through getInvalidFeeds. I went in and make use regex to chane all the \n to whitespace
reason: reason.replace(/\n/g, ' '),
Then I restart the images and ran again the results the regex do the work
A lot of debugging for second requirement
For the second requirement, I would check all the common point of the 304 invalid and it is weird to me that some of them seem to work fine and show up in telescope where it should not
An invalid post would be a post that we would not be able to pass through the parser.
I put the break point at src/backend when the queue is failed we will call invalidateFeed
And to my prediction, the rss-parser fetcha a feed and gets 304, the 304 has no body so we considered it to be failed request and mark it invalid
I try to make change at src/backend/feed/processor.js
if (!(response && response.status >= 400)) {
info.shouldDownload = false;
return info;
}
When I restart it, for a while there was no 304 psot which make me feel so exiciting about it but for a while the 304 keep showing up more and more so the problem has not been solved
Third fix
I actually found 2 resources which might be useful to help me in setting the ttl expire period for the key: Redis-doc and issue-100 and I wil be dig in to it in couple days to figure it out
Top comments (0)