Wigitize flaky due to Scaling Issues

on January 08, 2008

Update: Last 18 hours I have been combating performance issues because I got digged. Things seem to work more stable now, but still the background processing fails on me sometimes. Good lessons are being learned here so I will wrap that in a nice article soon!

Update: I just added another 1 GB slice, that should get us going for a while. Now just have to wait for the damn DNS entries to update.

I am preparing a new machine as we speak. Wow, did not to run into these luxury problems already! Thanks for the support!

If anyone thinks of buying a slicehost.com slice, please use this referral URL to donate:

https://manage.slicehost.com/customers/new?referrer=129028580

Machine 1 (256mb memory, how I started out with 2 days ago):

Machine 2 (1024mb memory) new

, ,

13 Responses to “Wigitize flaky due to Scaling Issues”

  1. January 8th, 2008 at 10:34 AM Cuneyt says:

    Nice article, and I, too, ran into scalability issues with Rails when trialling it vs. J2EE / Spring frameworks. Would love to see this framework improve in this regard (or ideally) have Java incorporate "convention" into their framework ala Java on Rails!

  2. January 8th, 2008 at 10:36 AM ynw says:

    Hmmm... you could avoid serving json. Let the javascript you provide pulling RSS directly from the original source. The magic will happen client side.

  3. January 8th, 2008 at 12:18 PM Dominiek says:

    Ynw: that is an interesting point, for now, that is a little too much Javascript foo for me :]

    In terms of performance serving the JSON is fast - directly through Apache. But aggregating and detecting the RSS is causing heavy background worker loads.

  4. January 8th, 2008 at 05:32 PM Harold says:

    I had a similar problem with a RSS aggregator I made. It would take forever to process them. I ended up switching it to the PHP based SimplePie for the aggregation and it was much faster.

  5. January 8th, 2008 at 07:49 PM Lee says:

    @Cuneyt You obviously haven't come across Grails then...

    www.grails.org

  6. January 8th, 2008 at 11:27 PM Frank says:

    Nice to see another slicer getting some traffic. However when "magic happening" you append the .... and if the site takes to long to finish the dots go off the screen.

  7. January 9th, 2008 at 06:48 AM slicematt says:

    Dominiek - let us know if you need any assistance, looks like you are doing great.

  8. January 9th, 2008 at 09:50 AM Marco Donati says:

    Sounds like you've got some good hosting plan. I'm thinking of transferring there too.

  9. January 10th, 2008 at 10:11 AM Jeff says:

    Great articles. What application are you using to display the load in those screenshots?

  10. January 12th, 2008 at 02:13 AM Alex Choo says:

    Hi.

    You might want to consider putting up your JS files on Amazon's S3. This will take a lot of stress off your servers when serving these files.

    Alex

  11. January 15th, 2008 at 08:17 AM Douglas F Shearer says:

    What CLI tool are you using for those screengrabs?

    Looks a lot friendlier that 'free' for at-a-glance viewing.

    Thanks.

  12. January 16th, 2008 at 01:03 PM Nicholas Orr says:

    I'm also wondering what CLI tool you are using in those screenshots, please tell :)

  13. January 17th, 2008 at 02:30 AM Mohammed says:

    htop

Sorry, comments are closed for this article.