Message Queue support

Discussion in 'Site Programming, Development and Design' started by stevozilik, Feb 8, 2011.

  1. Ray

    Ray

  2. Something a bit different, let me explain what I'm trying to achieve:

    1) My Asp.Net application is consuming a service that is pushing high volume of data to me (via ftp socket), approx. up to 100 messages/second.

    2) What I do with the data is
    a) Store it in the db.
    b) Expose a polling duplex service to my clients that consume that data in real time.

    Currently I'm doing this all whithin one app -> as data comes to web server, it's pushed to db and pushed to clients immediately. However I feel I can hit scalability issues as:
    a) I'm holding some of the received data in memory, If I had my web app running on multiple machines in paralell they would not see each other's memory...
    b) My polling duplex service does not scale, it can only handle a limited number of clients per application.


    If I put a Message Queue service in between getting the data (1) and working with it (2) then I could possibly read the queue and process it on different machines.

    Hope that's clearer now,
    Stevo
     
  3. Ray

    Ray

    I'm not sure what you mean by "on different machines". Can you elaborate on that a little further.
     
  4. I could be on the same box but different web application, or on a completely different box (within the same network, kind of a web-farm scenario like u guys -> i.e. you allow me to have up to X number of web apps but each of them could be potentionally sitting on a different place).

    The point is that they dont see each other and the Message Queue would be a common place for them to communicate. i.e. a common shared database could serve as message queue but that would not be a optimal solution in terms of speed, not even mentioning whether I can reference the same db from withing different web apps.
     
  5. Ray

    Ray

    No, I think that is where the misunderstanding lies. We are not on a web farm or any clustering environment. Basically we are on a stand alone web server we simply host several sites under one server. The only thing that is separated is the web server, mail server, and sql server. They are on different boxes to give our customers an added benefit of resources and security.
     
  6. That's fine I dont really care whether the different apps sit on one machine or not, still they dont see each other... So my only option would be database?
     
  7. Ray

    Ray

    Yes, I suspect so..
     
  8. I was looking for the exact same thing.

    It is certainly easy to use a database for this purpose, but that does not scale well at all.

    It would probably be worth it for Winhost to look into supporting a queuing technology, whether it be MSMQ or some other alternative.
     
    Last edited by a moderator: Oct 14, 2015

Share This Page