BUFFER OVERRUN - cause script stuck

Hello all !

I experiment some problems. I try to isolate them one by one.
With combinaison of some ingredients like input poling + servo moveto I get script stuck or servo delays. Sometime arduino down

http://a141.idata.over-blog.com/300x293/3/57/87/39/potion-magique.jpg

23:24:03.170 [i01.right.serial_outbox_0] WARN o.m.f.Inbox [Inbox.java:80] SwingGui inbox BUFFER OVERRUN dumping msg size 1025 - onRX
23:24:03.171 [COM3.portListener 2] WARN c.m.f.Outbox [Outbox.java:93] i01.left.serial outbox BUFFER OVERRUN size 1025

 

Thank for your help guys

calamity's picture

that kind of error happen

that kind of error happen when you have a service that produce more data than a listener service can process. 

I have run your script and did not get any error, but it said the word 'test' at a rate lower than what you asking for (it said test maybe a bit less than 1 time/s and you ask for 1-2 times/s). So your processor probably get busy saying 'test' and can't do other thing and at some point the buffers holding the messages between services overflow.

Try to not ask much than what your computer can do

moz4r's picture

Thank you Christian ! So t is

Thank you Christian ! So t is my hardware limits. 
Time to hunt cpu burning processes and increase some timers values

moz4r's picture

I have removed this line

I have removed this line inside the timer ( from the whole script ) , and increase time . it is better ! but still have buffer errors.

https://github.com/MyRobotLab/inmoov/blob/master/InmoovScript/inmoovLife...

it like the cpu doesn't like we change velocity every second while servo moving... there is something else it doesnt like I will look forward

calamity's picture

setVelocity have probably

setVelocity have probably nothing to do with your problem, it`s just a variable setting.

what kind of computer are you using?

moz4r's picture

It use a corei3 2ghz 4g

It use a corei3 2ghz 4g ram
dude I just saw my java 32bit. > update to 64 and reinstall all

no more buffer problem. I do more test tomorow
( i think i have an hardware problem too like a crappy ssd )

 

moz4r's picture

Worky ! multiple problems

Worky ! multiple problems identified

I can now use indecent timer values + moveTo + at same time indecent analog poling !

Now I wait the servo had finish the movement before sending another ( servo.isMoving() no worky ) :

if -0.1 <= i01.head.rothead.getCurrentPos()-i01.head.rothead.getPos() <= 0.1:i01.head.rothead.moveTo(random.uniform(60,120))

+ I use

head.rothead.enableAutoDetach(0) , I think the autoDetach(1) event cause buffer problem inside the timer

Thank for those functions, I continue playin with them :)

 

 

calamity's picture

Hi anthony there was

Hi anthony

there was effectively a problem with the autoDetach. The detach() call was place at the wrong place, wich cause a longer wait than expect, resulting in a bufferoverflow

I fix that in the latest version, along with the isMoving()

 

 

moz4r's picture

I m happy if we found this

I m happy if we found this bug !
I try to use autodetach() on 1953. I have no buffer error anymore. But I can't control velocity anymore ( full speed ) if autodetach(true)

calamity's picture

Nice found.... You discover

Nice found.... You discover that before the servo start moving it send a servoEventStopped... that event quickly detach the servo before he finish moving... On the next move, it attach and "jump" at the position he think he should be, and that jump is always at full speed.

fixed in the latest, flash  your arduino

But the good thing is that it show that it's working

moz4r's picture

Wahooo the mechanical is

Wahooo the mechanical is perfect now ! a real clock

If we meet one day I owe you a lot of

http://www.humourr.com/divers/images-humour/00041.jpg

 

moz4r's picture

Arg, sorry I spoke too fast.

Arg, sorry I spoke too fast. there is still a little bug, the last one I think :

If setVelocity=-1 or is not set , autodetach(1) cause a lot of bufferoverun.

tested with a simple script

calamity's picture

use serVelocity(-1) !!!!

use serVelocity(-1) !!!! why!!!  who want to use servo without speed control LOL

I will look at this as soon as I can

moz4r's picture

you right :) It can be

you right :)

It can be usefull for fullspeed if we don't know maxvelocity ( like jaw servo, or for newbies )

calamity's picture

I think I fix the autodetach

I think I fix the autodetach when velocity = -1

now you can try to break it again :)

moz4r's picture

buffer overrun crash

Hi dude. oups i did it again :)

I still have random buffer overrun crash if I use poling + servo.moveTo

. I try to isolate the problem but it is hard. I found this. Don't know if it is related but it is a good crash 100% reproductible :)

If I go inside arduino serial gui , and if I select text inside ( same problem if I click inside the terminal log )
Maybe sometime , something cause interferencies between gui and mrlcomm, like this test ?

 

import random
from time import sleep
arduino = Runtime.createAndStart("arduino","Arduino")
arduino.connect("COM5")
def publishPinLeft(pins):
 for pin in range(0, len(pins)):
  print 0
    
arduino.enablePin(6,100)

 

 

 

 

calamity's picture

Let me try to explain you how

Let me try to explain you how data are transfered to different services, it will help you to understand why the buffer overflow happen and how to try to avoid them

You have a service (A) that publish data (in your case, the Arduino service that publish the pin value), so the data are put into the arduino outbox buffer. Then those data will be transfered to the inbox buffer of every services (B) that listening to those data (in your case, the serialGui). Service B will then take and remove one data from it's inbox and process it, then take another one and process it and so on.

Now what happen if service A publish data more quickly than service B can process them, the data will accumulate in the input buffer of service B. At some point, the input buffer will get full and the output buffer of service A will complain that he have no place to put the data, that's the error message you are seeing.

So sometimes it need to have some synchronization between the publisher service and the listener service to avoid the buffer to overflow.

So how to prevent the buffer overflow

  • Have service A publish less data, it will gives more time for service B to process the data. (in your case you may try to lower the pin polling rate)
  • Have service B process the data faster by optimizing code
  • Have some code in service B that decide if it should process the data or discard them because it's not going fast enough (kind of a safety valve)

in your specific example, when you select text in the box, the serialGui seem to halt, no other data get processed and show in the GUI. So the input buffer get quickly full and you get the buffer overflow errors.

Not sure how to fix that specific case, but one thing that's i'm sure is that those buffer overflow have no consequence other than losing some data to show in the serialGui. If anothe service is listening to the publish data, it should receive the data normally just as the serialGui is working normally. So that's annoying, but your code should work normally

 

GroG's picture

I added a monitor button in

I added a monitor button in the Serial gui

There are several things which take time when data goes to the SerialGui

  • Gui is slow compared to faster memory things - displaying stuff on screen takes 'longer'
  • Unfortunately, Serial does not publish a byte array, but publishes every byte .. moving stuf piece by piece can take longer than sending a whole buffer ...
  • Gui also decodes and adds spaces and converts to decimal, adds carriage return when expected
  • Gui also checks for when its display is too big - trims its buffer and reprints :(

Lots of slowness with guis...  so monitor button is off now by default - does not show data

You also want to make sure your not logging all this stuff (logging takes time too)

runtime->logging->info or warn or error (not debug) 

:)

moz4r's picture

thank you ! those things made

thank you ! those things made me crazy, no more error or icerberg, see is calm now !