The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here
"taskScheduler loops in infinity on server with Loop Remote Files"
mugicagonzalez_
Member Posts: 14 Contributor I
Hello all!
I am using the "Loop Remote Files" from the Jackhammer extension. But irregularly it happens that the process runs into infinity (and doesn't time-out) on the server but doesn't loop the remote files anymore.
My gut feeling says it's due to an overloaded FTP server, but I wanted to check here first before I start moving to another FTP server.
Any hint on how I can investigate the real problem?
My server log file produces this in bulk:
I am using the "Loop Remote Files" from the Jackhammer extension. But irregularly it happens that the process runs into infinity (and doesn't time-out) on the server but doesn't loop the remote files anymore.
My gut feeling says it's due to an overloaded FTP server, but I wanted to check here first before I start moving to another FTP server.
Any hint on how I can investigate the real problem?
My server log file produces this in bulk:
<div>2019-03-06 08:41:28.510 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.RemoteRepositoryService : Trying to update JWT token from auth properties.</div><div>2019-03-06 08:41:28.510 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.RemoteRepositoryService : Skipped updating the JWT token as old one equals the new one.</div><div>2019-03-06 08:41:32.080 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:41:37.080 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:41:42.080 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:41:47.080 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:41:52.080 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:41:57.081 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:42:02.081 INFO 72445 --- [taskScheduler-4] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:42:07.081 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:42:12.081 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:42:17.082 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:42:22.082 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:42:27.082 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:42:28.510 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.RemoteRepositoryService : Trying to update JWT token from auth properties.</div><div>2019-03-06 08:42:28.510 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.RemoteRepositoryService : Skipped updating the JWT token as old one equals the new one.</div><div>2019-03-06 08:42:32.082 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:42:37.082 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:42:42.083 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:42:47.083 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:42:52.083 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:42:57.083 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:43:02.083 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:43:07.084 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:43:12.084 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:43:17.084 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:43:22.084 INFO 72445 --- [taskScheduler-1] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:43:27.084 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:43:28.510 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.RemoteRepositoryService : Trying to update JWT token from auth properties.</div><div>2019-03-06 08:43:28.510 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.RemoteRepositoryService : Skipped updating the JWT token as old one equals the new one.</div><div>2019-03-06 08:43:32.084 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:43:37.085 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:43:42.085 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:43:47.085 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:43:52.085 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:43:57.085 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:44:02.086 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:44:07.086 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:44:12.086 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:44:17.086 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:44:22.086 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:44:27.087 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:44:28.511 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.RemoteRepositoryService : Trying to update JWT token from auth properties.</div><div>2019-03-06 08:44:28.511 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.RemoteRepositoryService : Skipped updating the JWT token as old one equals the new one.</div><div>2019-03-06 08:44:32.087 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:44:37.087 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:44:42.087 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:44:47.087 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:44:52.088 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:44:57.088 INFO 72445 --- [taskScheduler-1] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:45:02.088 INFO 72445 --- [taskScheduler-1] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:45:07.088 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:45:12.089 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:45:17.089 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:45:22.089 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:45:27.089 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:45:28.511 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.RemoteRepositoryService : Trying to update JWT token from auth properties.</div><div>2019-03-06 08:45:28.511 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.RemoteRepositoryService : Skipped updating the JWT token as old one equals the new one.</div><div>2019-03-06 08:45:32.089 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:45:37.090 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:45:42.090 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:45:47.090 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:45:52.090 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:45:57.090 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:46:02.091 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:46:07.091 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:46:12.091 INFO 72445 --- [taskScheduler-2] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:46:17.091 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:46:22.092 INFO 72445 --- [taskScheduler-1] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:46:27.092 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:46:28.511 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.RemoteRepositoryService : Trying to update JWT token from auth properties.</div><div>2019-03-06 08:46:28.511 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.RemoteRepositoryService : Skipped updating the JWT token as old one equals the new one.</div><div>2019-03-06 08:46:32.092 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:46:37.092 INFO 72445 --- [taskScheduler-7] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:46:42.092 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:46:47.093 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:46:52.093 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:46:57.093 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:47:02.093 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:47:07.094 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:47:12.094 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:47:17.094 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:47:22.094 INFO 72445 --- [taskScheduler-1] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:47:27.094 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:47:28.511 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.RemoteRepositoryService : Trying to update JWT token from auth properties.</div><div>2019-03-06 08:47:28.512 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.RemoteRepositoryService : Skipped updating the JWT token as old one equals the new one.</div><div>2019-03-06 08:47:32.095 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:47:37.095 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:47:42.095 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:47:47.095 INFO 72445 --- [taskScheduler-5] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB</div><div>2019-03-06 08:47:52.095 INFO 72445 --- [taskScheduler-2] c.r.e.j.service.OutOfMemoryWatchdog : Current memory usage is 2272MB / 6144MB ....</div>
Tagged:
0
Best Answer
-
Burhan_OWC Member Posts: 2 Contributor IHi,
we have invastigated and isolated and fixed the issue. Your guess was right, in case of an overload of the server the operator would run into an infinite loop, since it could not set up the ftp enviroment in a correct manner and in order to not mess anything up on your remote file system (FTP server), it would keep failing during settiung up the evniroment and not be able to continue, hence run into an infinite loop.
Now the operator would simply fail, allowing you to handle the exception with the (HandleException Operator).
A release was made this morning and can be downloaded from the rapidminer marketplace (https://marketplace.rapidminer.com/UpdateServer/faces/product_details.xhtml?productId=rmx_toolkit).
Please make sure to update your extension to the latest version to resolve this issue.
Since there are many different FTP servers out there, you still might see issues for your specific ftp server. In that case just reach out to us (Old World Computing) again and we will resolve it.
kind regards
Burhan Yildiz6
Answers
I am part of the Old World Computing team (as a Student) and I would like to try and reproduce said bahavior to locate the issue and hopefully fix it.
But for that I am in need for additional Information, since the cause of the issue could be anything:
(Rapidminer menu: Extensions -> About installed Extensions-> About Jackhammer Extension)
It would be a big help if you could provide answers to as many of the above questions as you can, so I can investigate this issue in a quick manner.
kind regards
Burhan Yildiz
(Rapidminer menu: Extensions -> About installed Extensions-> About Jackhammer Extension)
2.1.784
9.1
No, and no access to config. But could experiment with another server though.
46.30.211.164:21
I manually stop the process and re-run it without any issue (often a couple of hours later, because I check the server executions once a day)