I believe I can provide some insights into the issue.
The problem stems from the configuration of the write buffer size inside the ethclient. In the new version of ethclient (available on GitHub at github.com/ethereum/go-ethereum/ethclient), the write buffer size is hardcoded to 1024 bytes. If the data being written exceeds this size, it will not function properly.
Conversely, the xdcpos version of ethclient (located at github.com/XinFinOrg/XDPoSChain/ethclient) uses an older version of ethclient that lacks this limitation on the write buffer size. Consequently, it works as expected.
It's important to note that this issue isn't the fault of the ethclient from go-ethereum. The root cause lies in our XDC node's handling of the WebSocket, which is somewhat outdated.
The latest ethclient is attempting to divide large payloads into smaller chunks, with each chunk having a maximum size of 1024 bytes. However, our XDC node lacks a mechanism to handle these chunked data. As a result, when the XDC node receives the first 1024 bytes, it attempts to decode them immediately, even though it's supposed to wait until all the chunks arrive. Because the data is chunked, decoding it prematurely will fail. This explains why you are encountering failed requests when the payload is larger.
To address this issue fundamentally, we are actively working on upgrading the entire WebSocket and its related RPC module in XDPoS.
In the meantime, as a temporary solution, I recommend keeping the batch size of 4 jobs when making requests.
I am very grateful for your comment. And what I want to ask is, when XDCPos2.0 starts on the mainnet, will this issue be resolved? This is very important for decentralized oracles on the XDC Network. It is absolutely necessary in order to sufficiently run jobs. This is also the case for projects that require oracles on the XDC Network.
Hi,
I believe I can provide some insights into the issue.
The problem stems from the configuration of the write buffer size inside the ethclient. In the new version of ethclient (available on GitHub at github.com/ethereum/go-ethereum/ethclient), the write buffer size is hardcoded to 1024 bytes. If the data being written exceeds this size, it will not function properly.
Conversely, the xdcpos version of ethclient (located at github.com/XinFinOrg/XDPoSChain/ethclient) uses an older version of ethclient that lacks this limitation on the write buffer size. Consequently, it works as expected.
You can view the relevant code at this link: github.com/ethereum/go-ethereum/bl...
It's important to note that this issue isn't the fault of the ethclient from go-ethereum. The root cause lies in our XDC node's handling of the WebSocket, which is somewhat outdated.
The latest ethclient is attempting to divide large payloads into smaller chunks, with each chunk having a maximum size of 1024 bytes. However, our XDC node lacks a mechanism to handle these chunked data. As a result, when the XDC node receives the first 1024 bytes, it attempts to decode them immediately, even though it's supposed to wait until all the chunks arrive. Because the data is chunked, decoding it prematurely will fail. This explains why you are encountering failed requests when the payload is larger.
To address this issue fundamentally, we are actively working on upgrading the entire WebSocket and its related RPC module in XDPoS.
In the meantime, as a temporary solution, I recommend keeping the batch size of 4 jobs when making requests.
Thank you so much for the detailed inputs!, this really helps
I am very grateful for your comment. And what I want to ask is, when XDCPos2.0 starts on the mainnet, will this issue be resolved? This is very important for decentralized oracles on the XDC Network. It is absolutely necessary in order to sufficiently run jobs. This is also the case for projects that require oracles on the XDC Network.
We have merged the PR into master branch. it's currently undergo testing on our testnet.
PR: github.com/XinFinOrg/XDPoSChain/pu...
It will be fixed/working before v2 consensus is even enabled
Thank you for your response, and I also appreciate the daily development efforts of the XDC team. I'm really looking forward to the new V2 consensus.