LS adapter gets stops automatically get function only after restart the LS

[rvkvino]

Hello,
As per your instruction I have checked the log, now my system didn't work got stopped the pushing rates so suddenly checked log. Please check this log I have attached,
07-Dec-20 04:37:21,331|INFO |L.requests |PUMP POOLED THREAD 1 |Closed session S91a4ff89e6ee459fM656T2306451 with internal cause code: 38/2 (Connection interrupted by client).
07-Dec-20 04:37:23,715|INFO |streamerMonitorText|Timer-1 |Total threads = 92, Total heap = 62914560 (free = 43428880), Sessions = 3 (max = 13, refused = 0), New sessions = [+2, -2], Connections = 3 (max = 18), New connections = [+4, -4], In-pool threads = 32, Active threads = 0, Available threads = 32, Queued tasks = 0, Pool queue wait = 0, NIO write queue = 0, NIO write queue wait = 0, NIO write selectors = 4, NIO total selectors = 16, Subscribed items = 12 (for 36 subscriptions), Inbound throughput = 0 updates/s (pre-filtered = 0), Outbound throughput = 0 updates/s (0.73 kbit/s, max = 50.97), Lost updates = 0 (total = 0), Total bytes sent = 26829363, Client messages throughput = 0 msgs/s (0 kbit/s, max = 0), Total messages handled = 0, Extra sleep = 0, Notify delay = 0
07-Dec-20 04:37:25,159|INFO |L.requests |SERVER POOLED THREAD 8 |Serving request: /lightstreamer/create_session.js --> LS_op2=create&LS_phase=8801&LS_cause=new.api&LS_po lling=true&LS_polling_millis=0&LS_idle_millis=0&LS _cid=pcYgxn8m8 feOojyA1T661j3g2.pz479h7m&LS_adapter_set=WLSTOCKLI ST_REMOTE&LS_container=lsc& on "Lightstreamer HTTP Server" from 1.22.25.136:58569
07-Dec-20 04:37:25,160|INFO |L.requests |SERVER POOLED THREAD 8 |Starting new session: S3e7c8f24186ea3cdM656T3725160 (javascript_client 7.0.4 build 1721) on "Lightstreamer HTTP Server" from 1.22.25.136:58569
07-Dec-20 04:37:25,450|INFO |L.requests |SERVER POOLED THREAD 8 |Serving request: bind_session -> LS_session=S3e7c8f24186ea3cdM656T3725160&LS_phase= 8803&LS_cause=loop1&LS_container=lsc& on "Lightstreamer HTTP Server" from 1.22.25.136:58570
07-Dec-20 04:37:25,450|INFO |L.requests |SERVER POOLED THREAD 8 |Attaching session: S3e7c8f24186ea3cdM656T3725160 on "Lightstreamer HTTP Server" from 1.22.25.136:58570
07-Dec-20 04:37:25,455|INFO |L.requests |SERVER POOLED THREAD 8 |Serving request: control -> LS_mode=COMMAND&LS_id=kgnbull&LS_schema=key%20comm and%20desc%20bid%20ask%20high%20low%20order%20deli very&LS_data_adapter=WLQUOTE_ADAPTER&LS_snapshot=t rue&LS_table=1&LS_req_phase=2&LS_win_phase=1&LS_op =add&LS_session=S3e7c8f24186ea3cdM656T3725160& on "Lightstreamer HTTP Server" from 1.22.25.136:58570
07-Dec-20 04:37:25,455|INFO |L.requests |SERVER POOLED THREAD 8 |Controlling session: S3e7c8f24186ea3cdM656T3725160 on "Lightstreamer HTTP Server" from 1.22.25.136:58570
07-Dec-20 04:37:25,457|INFO |L.requests |SERVER POOLED THREAD 4 |Controlling session: S3e7c8f24186ea3cdM656T3725160 on "Lightstreamer HTTP Server" from 1.22.25.136:58570
07-Dec-20 04:37:46,992|INFO |L.requests |PUMP POOLED THREAD 4 |Closed session S3e7c8f24186ea3cdM656T3725160 with internal cause code: 38/2 (Connection interrupted by client).
07-Dec-20 04:37:51,030|INFO |L.requests |SERVER POOLED THREAD 1 |Starting new session: S2c036881423e770bM656T3751030 (javascript_client 7.0.4 build 1721) on "Lightstreamer HTTP Server" from 1.22.25.136:58610
07-Dec-20 04:37:51,992|DEBUG|L.subscriptions |SERVER POOLED THREAD 4 |Unsubscribing from item kgnbull on WLSTOCKLIST_REMOTE.WLQUOTE_ADAPTER
07-Dec-20 04:37:51,992|DEBUG|L.subscriptions |SERVER POOLED THREAD 1 |Unsubscribing from item Marketstatus-kgnbull on WLSTOCKLIST_REMOTE.WLQUOTE_ADAPTER
07-Dec-20 04:37:56,468|INFO |L.requests |SERVER POOLED THREAD 1 |Serving request: /lightstreamer/bind_session.js --> LS_session=S2c036881423e770bM656T3751030&LS_phase= 6104&LS_cause=ws.error.failure.wsc.unknown&LS_cont ent_length=4000000&LS_container=lsc& on "Lightstreamer HTTP Server" from 1.22.25.136:58610
07-Dec-20 04:37:56,468|INFO |L.requests |SERVER POOLED THREAD 1 |Attaching session: S2c036881423e770bM656T3751030 on "Lightstreamer HTTP Server" from 1.22.25.136:58610
07-Dec-20 04:37:56,796|INFO |L.requests |SERVER POOLED THREAD 4 |Serving request: /lightstreamer/control.js --> LS_mode=COMMAND&LS_id=kgnbull&LS_schema=key%20comm and%20desc%20bid%20ask%20high%20low%20order%20deli very&LS_data_adapter=WLQUOTE_ADAPTER&LS_snapshot=t rue&LS_table=1&LS_req_phase=2&LS_win_phase=1&LS_op =add&LS_unique=1&LS_session=S2c036881423e770bM656T 3751030& on "Lightstreamer HTTP Server" from 1.22.25.136:58615
07-Dec-20 04:37:56,797|INFO |L.requests |SERVER POOLED THREAD 4 |Controlling session: S2c036881423e770bM656T3751030 on "Lightstreamer HTTP Server" from 1.22.25.136:58615
07-Dec-20 04:37:56,798|DEBUG|L.subscriptions |SERVER POOLED THREAD 1 |Subscribing to item kgnbull on WLSTOCKLIST_REMOTE.WLQUOTE_ADAPTER
07-Dec-20 04:37:57,105|DEBUG|L.subscriptions |SERVER POOLED THREAD 4 |Subscribing to item Marketstatus-kgnbull on WLSTOCKLIST_REMOTE.WLQUOTE_ADAPTER
07-Dec-20 04:38:02,614|INFO |L.requests |SERVER POOLED THREAD 1 |Serving request: /lightstreamer/control.js --> LS_op=destroy&LS_session=S84a38836513b0243M656T275 6818&LS_cause=new.timeout.10.1&LS_unique=84& on "Lightstreamer HTTP Server" from 169.149.227.120:35294
07-Dec-20 04:38:02,618|INFO |L.requests |SERVER POOLED THREAD 1 |Serving request: /lightstreamer/create_session.js --> LS_op2=create&LS_phase=42506&LS_cause=create.timeo ut&LS_polling=true&LS_polling_millis=0&LS_idle_mil lis=0&LS_cid=pcYgxn8m8 feOojyA1T661k3g2.pz479h8o&LS_adapter_set=WLSTOCKLI ST_REMOTE&LS_container=lsc& on "Lightstreamer HTTP Server" from 169.149.203.46:35296
07-Dec-20 04:38:02,622|INFO |L.requests |SERVER POOLED THREAD 1 |Starting new session: S10bbf993d030c7feM656T3802622 (javascript_client 7.0.5 build 1722) on "Lightstreamer HTTP Server" from 169.149.203.46:35296
07-Dec-20 04:38:03,273|INFO |L.requests |SERVER POOLED THREAD 1 |Serving request: /lightstreamer/bind_session.js --> LS_session=S10bbf993d030c7feM656T3802622&LS_phase= 42508&LS_cause=loop1&LS_keepalive_millis=5000&LS_c ontent_length=4000000&LS_container=lsc& on "Lightstreamer HTTP Server" from 169.149.203.46:35296
07-Dec-20 04:38:03,273|INFO |L.requests |SERVER POOLED THREAD 1 |Attaching session: S10bbf993d030c7feM656T3802622 on "Lightstreamer HTTP Server" from 169.149.203.46:35296
07-Dec-20 04:38:03,279|INFO |L.requests |SERVER POOLED THREAD 1 |Serving request: /lightstreamer/control.js --> LS_mode=MERGE&LS_id=GOLDFEB%20MGOLD-C%20MGOLD-F%20SILVERMAR%20SILVER-F%20MSILVER-C%20MSILVER-F%20SPOT-GOLD%20SPOT-SILVER%20SPOT-INR&LS_schema=desc%20bid%20ask%20high%20low%20ltp& LS_data_adapter=WLQUOTE_ADAPTER&LS_snapshot=true&L S_table=56&LS_req_phase=57&LS_win_phase=57&LS_op=a dd&LS_unique=85&LS_session=S10bbf993d030c7feM656T3 802622& on "Lightstreamer HTTP Server" from 169.149.227.120:35294
07-Dec-20 04:38:03,279|INFO |L.requests |SERVER POOLED THREAD 1 |Controlling session: S10bbf993d030c7feM656T3802622 on "Lightstreamer HTTP Server" from 169.149.227.120:35294
07-Dec-20 04:38:03,280|DEBUG|L.subscriptions |SERVER POOLED THREAD 3 |Subscribing to item MGOLD-F on WLSTOCKLIST_REMOTE.WLQUOTE_ADAPTER
07-Dec-20 04:38:03,280|DEBUG|L.subscriptions |SERVER POOLED THREAD 8 |Subscribing to item MSILVER-C on WLSTOCKLIST_REMOTE.WLQUOTE_ADAPTER
07-Dec-20 04:38:03,280|DEBUG|L.subscriptions |SERVER POOLED THREAD 4 |Subscribing to item SILVER-F on WLSTOCKLIST_REMOTE.WLQUOTE_ADAPTER
07-Dec-20 04:38:03,659|INFO |L.requests |SERVER POOLED THREAD 4 |Serving request: /lightstreamer/control.js --> LS_mode=MERGE&LS_id=GOLDFEB%20MGOLD-C%20MGOLD-F%20SILVERMAR%20SILVER-F%20MSILVER-C%20MSILVER-F%20SPOT-GOLD%20SPOT-SILVER%20SPOT-INR&LS_schema=desc%20bid%20ask%20high%20low%20ltp& LS_data_adapter=WLQUOTE_ADAPTER&LS_snapshot=true&L S_table=57&LS_req_phase=58&LS_win_phase=57&LS_op=a dd&LS_unique=86&LS_session=S10bbf993d030c7feM656T3 802622& on "Lightstreamer HTTP Server" from 169.149.227.120:35294
07-Dec-20 04:38:03,660|INFO |L.requests |SERVER POOLED THREAD 4 |Controlling session: S10bbf993d030c7feM656T3802622 on "Lightstreamer HTTP Server" from 169.149.227.120:35294
07-Dec-20 04:38:11,243|INFO |L.requests |PUMP POOLED THREAD 4 |Closed session S10bbf993d030c7feM656T3802622 with internal cause code: 38/5 (Connection interrupted by client).
07-Dec-20 04:38:16,243|DEBUG|L.subscriptions |SERVER POOLED THREAD 3 |Unsubscribing from item MGOLD-F on WLSTOCKLIST_REMOTE.WLQUOTE_ADAPTER
07-Dec-20 04:38:16,244|DEBUG|L.subscriptions |SERVER POOLED THREAD 3 |Unsubscribing from item SILVER-F on WLSTOCKLIST_REMOTE.WLQUOTE_ADAPTER
07-Dec-20 04:38:16,244|DEBUG|L.subscriptions |SERVER POOLED THREAD 5 |Unsubscribing from item MSILVER-C on WLSTOCKLIST_REMOTE.WLQUOTE_ADAPTER
07-Dec-20 04:39:23,762|INFO |streamerMonitorText|Timer-1 |Total threads = 93, Total heap = 62914560 (free = 41810672), Sessions = 4 (max = 13, refused = 0), New sessions = [+3, -2], Connections = 4 (max = 18), New connections = [+10, -9], In-pool threads = 32, Active threads = 0, Available threads = 32, Queued tasks = 0, Pool queue wait = 0, NIO write queue = 0, NIO write queue wait = 0, NIO write selectors = 4, NIO total selectors = 16, Subscribed items = 12 (for 36 subscriptions), Inbound throughput = 0 updates/s (pre-filtered = 0), Outbound throughput = 0 updates/s (0.98 kbit/s, max = 50.97), Lost updates = 0 (total = 0), Total bytes sent = 26839721, Client messages throughput = 0 msgs/s (0 kbit/s, max = 0), Total messages handled = 0, Extra sleep = 0, Notify delay = 0

I have uploaded full log file in this link http://post.nammauzhavan.com/logs/Lightstreamer.log

[giuseppe.corti]

As you can see from the log, the messages reporting received updates from the Data Adapter ("INCOMING DATA for ... ") disappear completely from 03:50:01,925; but the rest of the server activity continues regularly.
New client sessions are processed properly, as well as subscriptions and unsubscription requests; just no new data from the Data Adapter were received.

Please you should double check your Data Adapter code and your back-end source of the data to try to figure out where could be addressed the stoppage.

Regards,
Giuseppe

[rvkvino]

But when I restart the LS and adapter this getting run normally. This is too hard to find out the issue.

[giuseppe.corti]

Indeed, in this case I would start the investigation from the initialization operations performed in the Adapters, since they seem to fix the issue.

Maybe you have a forever loop that once interrupted is not re-started?
Or a connection to an external service that as well is not restored following a failure?

[rvkvino]

Hello,
I have attached my full source code in this link http://post.nammauzhavan.com/logs/src.7z Could you please check and help me on this.

[giuseppe.corti]

Hi rvkvino,

I have just taken a quick review of your code, and indeed I cannot provide a definitive answer but some tips to continue the investigation.
I think you should start by monitoring specifically the getJSON method, that if I am right, is called repeatedly to retrieve data updates from the backend.
I would add some logging in the method; I noticed that in the code you use System.out that should be ok if you have actually access to the console output of the running adapters process.
Just some basic text message in order to verify that the getJSON it is regularly called up for the entire life of the process.

Furthermore, just as a precaution I would add a generic catch to Exception, always in the getJSON method, because this could be the case where an unhandled exception stops the flow of data to the Lightstreamer server.

Regards,
Giuseppe

[rvkvino]

Hello,
As per your advice I have added the generic catch to Exception and update the adapter and now it's running. I will monitor 2 days and update the feedback.