Ewon TechForum
Providing technical solutions to technical requests


Is there a request size limit for the web2m api? cannot download files over 22mb
sommmen Offline
Junior Member
#1
Hi,

I am downloading .zip files via the web2m api. These zip files range from a couple mb's to about a gigabyte.
However when i download these files it seems that it gets stuck around 22,200 kb (~22mb).

if i use the vpn it works just fine. This means it must be the web2m connection.

Some photos describing the problem:

https://docs.google.com/document/d/1j3VE...sp=sharing

So:

1. Is there a content limit set for GET calls for the web2m api?
2. Can this limit be set higher? to e.g. 5gb would fit my particular usecase.
Reply

simon Offline
eWON Support
#2
Hi,

I don't think there is a size limit but perhaps a time limit ( or maybe a limit in the Ewon. Do you use CD/Cosy141 or Cosy131/Flexy ?)
However, about your question to download 5GB, do not forget that the traffic you generate with the API is taken in account in the billing.
As a reminder :
Free+ Account : 1GB of traffic/month
Pro Account : 6 GB of traffic/month (extra traffic is charged on your credit)
So I think Talk2M is not really the right solution in that case.

Simon
Reply

sommmen Offline
Junior Member
#3
(13-11-2019, 11:07 AM)simon Wrote: Hi,

I don't think there is a size limit but perhaps a time limit ( or maybe a limit in the Ewon. Do you use CD/Cosy141 or Cosy131/Flexy ?)
However, about your question to download 5GB, do not forget that the traffic you generate with the API is taken in account in the billing.
As a reminder :
Free+ Account : 1GB of traffic/month
Pro Account : 6 GB of traffic/month (extra traffic is charged on your credit)
So I think Talk2M is not really the right solution in that case.

Simon

Hi Simon,

The ewon is a flexy 205. We have a pro account and we're aware that this is gonna cost us, it does not matter however.
If i send the files over a vpn connection I will also pay.

Do you know what this time limit is?
I will also check the ewon for any limits. From the machin's perspective (the remote location) al seems good. It seems that the vpn cloud downloads the file completely.
Reply

simon Offline
eWON Support
#4
Hi,

I got the confirmation that the timeout is 10 min.

Simon
Reply

sommmen Offline
Junior Member
#5
(13-11-2019, 12:31 PM)simon Wrote: Hi,

I got the confirmation that the timeout is 10 min.

Simon

Hi Simon - thanks for figuring this out for me. your efforts are appreciated.

Maybe it would be wise to add this limitation of the web2m api to the documentation the api reference document (RG-0004-00 en-US ENGLISH)

I also checked the EWON but found no timeout/rate settings.

Do you reckon that this timeout of 10 minutes could be set to a larger time? Could you ask your team?
I mean if the connection is active, meaning packets are sent and received - it shouldn't close the connection just cause 10 minutes have passed right?
Only when there is no activity the connection could be closed.

As an alternative to web2m i was looking into FTP:
https://developer.ewon.biz/content/ftp-gateway
However this downloads the file to the ewon first and then sends it through. The flexy only has 30mb of user storage so that is too few for my use case.

I think there is only 2 ways to go about this, and that is to either have you guys move the time limit higher (30m?) or i would have to cut the files into smaller pieces.
That would however require a lot of extra logic/complexity from me, on both the remote site and the headquarters where i'm downloading the files to.
Reply

simon Offline
eWON Support
#6
Hi,

Regarding the timeout, I will check with the team and let you know.

Regarding the FTP Option, you can use a SD card formatted in ext3 to extend the usr/ partition : https://ewon.biz/docs/librariesprovider1...memory.pdf

Simon
Reply

sommmen Offline
Junior Member
#7
(13-11-2019, 07:26 PM)simon Wrote: Hi,

Regarding the timeout, I will check with the team and let you know.

Regarding the FTP Option, you can use a SD card formatted in ext3 to extend the usr/ partition : https://ewon.biz/docs/librariesprovider1...memory.pdf

Simon

Hi Simon, I just ran a test with curl a couple times, but it seems the time/byte limit is set to 00:03:04 / 22mb.
This means the timeout limit of 10 minutes seems a bit wrong.

https://docs.google.com/document/d/1j3VE...sp=sharing

(13-11-2019, 07:26 PM)simon Wrote: Hi,

Regarding the timeout, I will check with the team and let you know.

Regarding the FTP Option, you can use a SD card formatted in ext3 to extend the usr/ partition : https://ewon.biz/docs/librariesprovider1...memory.pdf

Simon

Thanks for the info on the SD card. unfortunately we already have a couple sites up an running, so i can't easily get an SD card in there.
Reply

sommmen Offline
Junior Member
#8
As a sidenote, tried using the Ranges header (my server supports it) but that did not work either via the web2m api.

https://developer.mozilla.org/en-US/docs...e_requests
Reply

simon Offline
eWON Support
#9
Hi,

One thing. I think that during your tests, you were actually limited to the Web server timeout of the Ewon (SYS parameter HTTP_REQTO) which is by default set to 3 min.
Can you try to set it to 20 min for example and try again ? (Go Setup > System > Storage > Edit SYSTEM config)

Simon
Reply

sommmen Offline
Junior Member
#10
(15-11-2019, 06:06 PM)simon Wrote: Hi,

One thing.  I think that during your tests, you were actually limited to the Web server timeout of the Ewon (SYS parameter HTTP_REQTO) which is by default set to 3 min.
Can you try to set it to 20 min for example and try again ? (Go Setup > System > Storage > Edit SYSTEM config)

Simon

Hi Simon,

I will check that out in a second. In the meantime, i reverted to creating an api similar to the http 'accept-ranges' header, which will split the file into chunks and download each chunk (e.g. bytes 0 to 1500, then bytes 1500 to 3000). I set the chunk size to be 10mb, meaning about 1:23s (sometimes, it miraculously goes to something like 0:50s btw).

Also - i get 0 notifications about this forum, so when you posted an answer or something i am not really notified about that.
I looked into the settings panel but found no settings regarding this.

(15-11-2019, 06:06 PM)simon Wrote: Hi,

One thing.  I think that during your tests, you were actually limited to the Web server timeout of the Ewon (SYS parameter HTTP_REQTO) which is by default set to 3 min.
Can you try to set it to 20 min for example and try again ? (Go Setup > System > Storage > Edit SYSTEM config)

Simon

Hi,

Just tried to set the HTTP_REQTO field, however as is it's completely empty.
Now what should i exactly  Put in? is the value in minutes? so just put in 20? or is it in seconds so 1200?
Reply

simon Offline
eWON Support
#11
Hi,

Actually I was wrong, the HTTP_REQTO si only relevant when you download files from the Ewon itself, not from a LAN device as you are doing... :-(

I have then no work-around to suggest, sorry.

[To be notified, you have to subscribe to the thread by clicking the link at the bottom left]

Simon
Reply

sommmen Offline
Junior Member
#12
(21-11-2019, 02:46 PM)simon Wrote: Hi,

Actually I was wrong, the HTTP_REQTO si only relevant when you download files from the Ewon itself, not from a LAN device as you are doing... :-(

I have then no work-around to suggest, sorry.

[To be notified, you have to subscribe to the thread by clicking the link at the bottom left]

Simon

Hi Simon,

In that case my chunk-api will work fine for now.
We'll likely be moving this solution to a different one in the future anyways.
I was just at SPS-Drive in Nürnberg and already saw some cool solutions from HMS at their booth.
Reply

simon Offline
eWON Support
#13
Hi,

Ok, Thanks for the feedback.
Let me know if you need help to choose your next solution (if it is still with Ewon ;-) )


Simon
Reply

sommmen Offline
Junior Member
#14
(03-12-2019, 04:20 PM)simon Wrote: Hi,

Ok, Thanks for the feedback.
Let me know if you need help to choose your next solution (if it is still with Ewon ;-) )


Simon

Yes Simon, this will be in some time since this solution works perfectly for now.

Also loving the new forum! I also got got notified by email directly which I Liked.
the new design is refreshing.

small sidenote: On Chrome the footer image seems a bit off;
https://imgur.com/53U0z6D
Reply

Ludo Offline
eWON Support
#15
Sidenote fixed Wink
Reply



Possibly Related Threads…
28-11-2019, 11:01 AM
Last Post: sommmen
04-03-2019, 07:03 PM
Last Post: simon
29-05-2018, 10:38 AM
Last Post: simon
23-11-2017, 10:58 AM
Last Post: simon
15-06-2017, 10:39 AM
Last Post: simon



Users browsing this thread:
1 Guest(s)



Theme © Ewon 2019 - Forum software by © MyBB - Cookie policy