I forgot to say that I'm running the Vivace edition, quite important due to bandwidth restrictions I guess.
I forgot to say that I'm running the Vivace edition, quite important due to bandwidth restrictions I guess.
Hi,
please, could you modify your lightstreamer_log_conf.xml file to set the "subscriptions" and "pump" categories to DEBUG level and then attach a server log showing the issue?
Sure.
I'm currently trying to simulate a trading environment. I've deployed an adapter that contains an embedded rate generator. This generator creates rates at variable speeds, i.e. Some rates are generated more frequently than others. My Lighstreamer adaptor listens to that generator and each time it receives a rate pushes it to the client side.
I have been doing some tests and also looking at the log files and now the problem seems clear to me (of course, probably it's not a problem). Basically I'm pushing all the rates objects through the same channel ("rates"). So, in the log file you can see how some currency pair rates are being discarded. See attached file "first-try.txt". In the client, I only receive the data that is pumped in the event.
So I guess I should create a channel per currency pair. Does this have any consequence in the communication? (i.e. does this mean more sockets open? does this mean more bandwidth usage? etc.)
Thanks for the useful help and regards,
Martin
PS. Note that I have stripped down the log for being able to upload it.
Hi again.
I have upgraded my test to create six different subscription channels one for each currency pair I have.
However, I'm now puzzled because the adapter seems to successfully send all the rates in the currency pairs some times, but there are other times that it performs throttling and only sends some of the rates. In the attached file "second-try.txt" see how the first time only one rate is sent, but in the second time the four rates are sent, the next time only 2 of five rates are sent, the next time 4/7, etc.
I'm sure there is some logic behind of this behaviour. Any tips?
Thanks,
Martin
Hello,
Are you using the MERGE subscription mode right?
The problem is that since the updates are all published on the same item, Lightstreamer merges the updates if needed:
In the example 6 updates arrives to the server but only one is forwarded to the clients.Code:08.May.07 09:02:05,934 <TRACE> INCOMING DATA for rates --> {bidValue=0.07943651574153077, sellCCY=USD, bidCCY=EUR, sellValue=0.19836981819406774} 08.May.07 09:02:05,935 <TRACE> INCOMING DATA for rates --> {bidValue=0.04185744998798879, sellCCY=JPY, bidCCY=EUR, sellValue=0.002155323147410959} 08.May.07 09:02:05,936 <TRACE> INCOMING DATA for rates --> {bidValue=0.9033005599346077, sellCCY=USD, bidCCY=JPY, sellValue=0.3281485803285823} 08.May.07 09:02:05,937 <TRACE> INCOMING DATA for rates --> {bidValue=0.947976333651295, sellCCY=GBP, bidCCY=EUR, sellValue=0.38439026792729647} 08.May.07 09:02:06,044 <TRACE> INCOMING DATA for rates --> {bidValue=0.9040453255654483, sellCCY=USD, bidCCY=EUR, sellValue=0.9458336664422791} 08.May.07 09:02:06,045 <TRACE> INCOMING DATA for rates --> {bidValue=0.9708380297991672, sellCCY=GBP, bidCCY=EUR, sellValue=0.215475738803115} 08.May.07 09:02:06,134 <TRACE> Pumping event in session S1321751661N1: 1,1,1||0.9708380297991672||0.215475738803115
If yoo take a look at the timings of the "Pumping in Session" messages you will see a 200ms pattern:
09:02:05,537
09:02:05,734
09:02:05,934
09:02:06,134
and so on...
Those 200ms are set in the Lightstreamer_conf.xml file:
Code xml:
<max_delay_millis>200</max_delay_millis>
What appens is that Lightstreamer is enabled to send just one packet each 200ms (this is a session related delay) unless there is too much data to be sent in one single packet (due to frequency limits bandwidth limits etc this is just a simple explanation of the delivering algorithm). Btw since you publish every update to the same item, after 200ms from the last "Pumping event in session" - from Lightstreamer point of view - there is only one relevant update that must be sent (the latest).
This is indeed a good idea.Originally Posted by churrusco
The socket is always the same as Lightstreamer protocol take care of handling more items on the same data flowOriginally Posted by churrusco
Obviously more bandwidth will be used if more updates reach the client, btw you can limit the bandwidth both client and server side.Originally Posted by churrusco
Hi,
You wrote another post while I was writing the response for the previous one
In the new log you can see how the merge algorithm works when different items are used:
Inside this 200ms you can see that the server accumulated 7 events but 4 are sent to the client. If you look further you can see that those 7 updates pertains to only 4 items because there are 3 EUR/GBP and 2 EUR/USD events so that the server to send the latest update per each item have to send only 4 updates.Code:08.May.07 09:56:57,239 <TRACE> INCOMING DATA for EUR/GBP --> {bidValue=0.3952132682553846, sellCCY=GBP, bidCCY=EUR, sellValue=0.1582634326389245} 08.May.07 09:56:57,338 <TRACE> INCOMING DATA for EUR/USD --> {bidValue=0.9052316638725739, sellCCY=USD, bidCCY=EUR, sellValue=0.47263148734258653} 08.May.07 09:56:57,339 <TRACE> INCOMING DATA for EUR/GBP --> {bidValue=0.7117531450899011, sellCCY=GBP, bidCCY=EUR, sellValue=0.5475364773823609} 08.May.07 09:56:57,340 <TRACE> INCOMING DATA for USD/NZD --> {bidValue=0.3461941736979528, sellCCY=NZD, bidCCY=USD, sellValue=0.19362361881184487} 08.May.07 09:56:57,439 <TRACE> INCOMING DATA for EUR/USD --> {bidValue=0.7626206088735283, sellCCY=USD, bidCCY=EUR, sellValue=0.5145519477153373} 08.May.07 09:56:57,440 <TRACE> INCOMING DATA for JPY/USD --> {bidValue=0.204416519895124, sellCCY=USD, bidCCY=JPY, sellValue=0.2764089416286716} 08.May.07 09:56:57,440 <TRACE> INCOMING DATA for EUR/GBP --> {bidValue=0.5949231423651182, sellCCY=GBP, bidCCY=EUR, sellValue=0.5972830556785825} 08.May.07 09:56:57,450 <TRACE> Pumping event in session S-725498679N1: 4,1,1||0.7626206088735283||0.5145519477153373 3,1,1||0.5949231423651182||0.5972830556785825 6,1,1|USD|0.3461941736979528|NZD|0.19362361881184487 2,1,1||0.204416519895124||0.2764089416286716
Hope that helps.
Absolutely it helps, and it makes quite a lot of sense indeed. Great feature and quite easy to configure. Actually it was my fault haven't re-read the documentation. I read it one or two months ago and almost forgot everything about these modes. I believe it has to do with the age
Happily enough the simulation is now working fine so I can start stressing it Also I would like to comment some minor things:
- I found that with the RAW mode the server log messages are weird as it seems as if Lighstreamer was throttling messages. But the reality is that all of them arrive at the client even when in the logs it seems that only one is pushed. See attached file.
- I also see that there is a timeout and the session is maintained idle for 30 seconds. I guess this feature is to optimize AJAX communication, isn't it? (I say it because we are targeting an Applet environment so our ideal is just a "0" timeout with no back buttons and this browser stuff.)
Cheers,
Martin
Hi Mone,
Just forget the first minor issue. I just have realized the obvious, that there are different pump threads and this is not a sequential process
Cheers,
Martin
Bookmarks