Developers Forum for XinFin XDC Network: xu zhaolin

Weekly Update

xu zhaolin — Fri, 28 Oct 2022 09:26:58 +0000

Dear All Community Members
Currently we have finished xsswap and supporting globiance by building sdk. In addition, we are trying to add XinFin and Apothem in metamask. Below are what we have done this week.Feel free to comment.

1 Hummingbot
Swith to xsswap from xdcswap:
https://github.com/Carry-So/xsswap-sdk
https://www.npmjs.com/package/xsswap-sdk
Create globiance-sdk v1.0.0-beta:
https://github.com/Carry-So/globiance-sdk
https://www.npmjs.com/package/globiance-sdk
Support globiance exchange on xinfin chain execpt WXDC token: symbol conflict with another WXDC token
Refactor documents: https://github.com/Carry-So/hummingbot/blob/xdc-release-1.8.0/README.md
1.2 MetaMask
Read documents
Setup development environment
2) Doing
2.1 MetaMask
Add xinfin and apothem as build-in network
2.2 Hummingbot
handle symbol conflict of WXDC: add 0x8A3cc832Bb6B255622E92dc9d4611F2A94d200DA to token list as WXDC2, pro100skm/xdc-token-list#3
3) Plan
3.1 MetaMask
Add xinfin and apothem as build-in network
Support xdc-prefix address
3.2 Hummingbot
Support globiance exchange: waiting for new contracts of testnet
Submit PR to hummingbot

Graphnode Xinfin Compatibility Project （Done）

xu zhaolin — Tue, 16 Aug 2022 06:52:53 +0000

Dear all，

We have finished all the work that needed for graphnode to be compatible with XinFin netwwork. You can duplicate our steps to get the same results!
**
We have modified some of the graph source code to make it compatible with xdc network! you must use the code we provide to implement our method.**

https://github.com/gzliudan/graph-node
Graphnode overview,we have modifeid it to be compatitable with the xdc!!!!

https://github.com/gzliudan/parity-common
gzliudan/parity-common: Collection of crates used in Parity projects

https://github.com/gzliudan/bad-token-subgraph
The subgraph for XRC20 on blockchain XinFin and Apothem

https://github.com/gzliudan/stake-subgraph
The subgraph for StakeContract on blockchain XinFin and Apothem

Step-by-Step tutorial

Deploy the Graph

Deploy three daemons：IPFS, PostgreSql, and graph-indexer. Ipfs and graph-indexer should run with the current user right.

1、Install IPFS

full tutorial： https://docs.ipfs.io/install/command-line/#official-distributions

1.1 Modify UDP Receive Buffer Size

Tutorial： https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size

sudo sysctl -w net.core.rmem_max=2500000
echo "net.core.rmem_max=2500000" | sudo tee -a /etc/sysctl.conf

1.2 Instal IPFS

Tutorial： https://docs.ipfs.io/install/command-line/#official-distributions

wget https://dist.ipfs.io/kubo/v0.14.0/kubo_v0.14.0_linux-amd64.tar.gz
tar -xvzf kubo_v0.14.0_linux-amd64.tar.gz
cd kubo
sudo bash install.sh
ipfs --version

1.3 Initialize IPFS

export IPFS_PATH="/var/lib/ipfs"
echo 'export IPFS_PATH="/var/lib/ipfs"' >> ${HOME}/.bashrc

USER="`whoami`"
sudo mkdir -p /var/lib/ipfs
sudo chown -R ${USER}:${USER} ${IPFS_PATH}
sudo chmod ug+rwx ${IPFS_PATH}

ipfs init
cp -p ${IPFS_PATH}/config ${IPFS_PATH}/config.bak
sed -i "s#/ip4/127.0.0.1/tcp/5001#/ip4/0.0.0.0/tcp/5001#g" ${IPFS_PATH}/config
ipfs config --json API.HTTPHeaders.Access-Control-Allow-Origin '["http://192.168.2.162:5001", "http://127.0.0.1:5001"]'
ipfs config --json API.HTTPHeaders.Access-Control-Allow-Methods '["PUT", "POST"]'

1.4 Creat Service

sudo tee /etc/systemd/system/ipfs.service <<EOF
[Unit]
Description=InterPlanetary File System (IPFS) daemon
Documentation=https://docs.ipfs.tech/
After=network.target
Requires=network.target

[Service]
Type=simple
User=${USER}
Group=${USER}
Environment="IPFS_PATH=${IPFS_PATH}"
Environment="IPFS_FD_MAX=8192"
LimitNOFILE=8192
MemorySwapMax=0
# TimeoutStartSec=infinity
WorkingDirectory=${IPFS_PATH}
ExecStart=/usr/local/bin/ipfs daemon
RestartSec=10
Restart=always
KillSignal=SIGINT
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=default.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable ipfs
sudo systemctl start ipfs
sudo systemctl status ipfs
curl -X POST http://127.0.0.1:5001/api/v0/version
# response: {"Version":"0.14.0","Commit":"","Repo":"12","System":"amd64/linux","Golang":"go1.18.3"}

2、Install PostgreSQL

Tutorial： https://www.postgresql.org/download/linux/ubuntu/

database username: graph
password: daniel2022
listener ports: 5432
database name: apothem_db、xinfin_db

2.1 Modify the kernel parameter

cat /proc/sys/vm/swappiness
sudo bash -c "sysctl -w vm.swappiness=1"
echo "vm.swappiness=1" | sudo tee -a /etc/sysctl.conf
cat /proc/sys/vm/swappiness

2.2 Install language support(i use Chinese here)

sudo apt install -y language-pack-zh*

2.3 Install software

# Create the file repository configuration:
sudo sh -c 'echo "deb [arch=$(dpkg --print-architecture)] http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'

# Import the repository signing key:
# wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo tee /etc/apt/trusted.gpg.d/postgresql.asc

# Update the package lists:
sudo apt update

# Install PostgreSQL 14
sudo apt install -y postgresql-14 postgresql-client-14 postgresql-server-dev-14 libpq-dev

2.4 Backup config file

cd /etc/postgresql/14/main
sudo cp -p pg_hba.conf pg_hba.conf.`date +%Y%m%d-%H%M%S`
sudo cp -p postgresql.conf postgresql.conf.`date +%Y%m%d-%H%M%S`

2.5 Modify pg_hba.conf

sudo tee /etc/postgresql/14/main/pg_hba.conf <<-\EOF
# Database administrative login by Unix domain socket
local   all             postgres                                peer

# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     md5
# IPv4 local connections:
host    all             all             127.0.0.1/32            md5
# IPv6 local connections:
host    all             all             ::1/128                 md5
# Allow replication connections from localhost, by a user with the
# replication privilege.
local   replication     all                                     peer
host    replication     all             127.0.0.1/32            md5
host    replication     all             ::1/128                 md5
EOF

2.6 Some parameters adjustments

sudo tee /etc/postgresql/14/main/conf.d/liudan.conf <<-\EOF
listen_addresses = '*'
port = 5432

tcp_keepalives_idle = 300
tcp_keepalives_interval = 20
tcp_keepalives_count = 9

log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log'
log_line_prefix = '%m:%r:%u@%d:[%p]: '
log_rotation_age = 1d
lc_messages = 'en_US.UTF-8'            # log is shown in English

shared_preload_libraries = 'pg_stat_statements'
EOF

2.7 change os user's postgres password

echo postgres:daniel2022 | sudo chpasswd

2.8 start the database

# pg_ctlcluster 14 main start
sudo systemctl daemon-reload
sudo systemctl restart postgresql@14-main
systemctl status postgresql@14-main

2.9 change database user's postgres password

su - postgres
psql -c "ALTER ROLE postgres WITH PASSWORD 'daniel2022';"
exit

2.10 Creat Database

create user of the database: graph
create database： apothem_db and xinfin_db

PGPASSWORD=daniel2022 psql -X -h 127.0.0.1 -U postgres postgres <<-\EOF
CREATE USER graph WITH PASSWORD 'daniel2022' nocreatedb;
CREATE DATABASE apothem_db WITH OWNER=graph TEMPLATE=template0 LC_COLLATE='zh_CN.UTF8' LC_CTYPE='zh_CN.UTF8' ENCODING='utf8';
CREATE DATABASE xinfin_db WITH OWNER=graph TEMPLATE=template0 LC_COLLATE='zh_CN.UTF8' LC_CTYPE='zh_CN.UTF8' ENCODING='utf8';
EOF

2.11 Create extension

create extension： pg_trgm、pg_stat_statements、btree_gist、postgres_fdw

2.11.1 testnet apothem

PGPASSWORD=daniel2022 psql -X -h 127.0.0.1 -U postgres apothem_db <<-\EOF
CREATE EXTENSION pg_trgm;
CREATE EXTENSION pg_stat_statements;
GRANT ALL ON FUNCTION pg_stat_statements_reset TO graph;
CREATE EXTENSION btree_gist;
CREATE EXTENSION postgres_fdw;
GRANT USAGE ON FOREIGN DATA WRAPPER postgres_fdw TO graph;
EOF

2.11.2 production net xinfin

PGPASSWORD=daniel2022 psql -X -h 127.0.0.1 -U postgres xinfin_db <<-\EOF
CREATE EXTENSION pg_trgm;
CREATE EXTENSION pg_stat_statements;
GRANT ALL ON FUNCTION pg_stat_statements_reset TO graph;
CREATE EXTENSION btree_gist;
CREATE EXTENSION postgres_fdw;
GRANT USAGE ON FOREIGN DATA WRAPPER postgres_fdw TO graph;
EOF

2.12 setting up environment for psql ：

cat >> ${HOME}/.bashrc <<-\EOF
export PGHOST=127.0.0.1
export PGPORT=5432
export PGDATABASE=apothem_db
export PGUSER=graph
export PGPASSWORD=daniel2022
EOF

source ${HOME}/.bashrc

cat > ${HOME}/.psqlrc <<-\EOF
\set PROMPT1 '%`date +%H:%M:%S` (%n@%M:%>)%/%R%#%x '
\set PROMPT2 '%M %n@%/%R%# '
\timing on
EOF

3、Install graph-node

Tutorial：

https://github.com/graphprotocol/graph-node/blob/master/docs/config.md
https://github.com/graphprotocol/graph-node/blob/master/node/resources/tests/full_config.toml
Running a Graph Node on Moonbeam: https://docs.moonbeam.network/node-operators/indexer-nodes/thegraph-node/
Deploy and Configure Graph-node: https://docs.thegraph.academy/official-docs/indexer/testnet/graph-protocol-testnet-baremetal/3_deployandconfiguregraphnode

3.1 Create log files directory

USER="`whoami`"
sudo mkdir -p /var/log/graph
sudo chown -R ${USER}:${USER} /var/log/graph

3.2 Create config files directory

USER="`whoami`"
sudo mkdir -p /etc/graph-node
sudo chown -R ${USER}:${USER} /etc/graph-node
touch /etc/graph-node/expensive-queries.txt

3.3 Install Rust

安装或者更新:

cargo -V
if [ "$?" == 0 ] ; then
    rustup update stable
else
    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
    source ${HOME}/.cargo/env
fi
cargo -V

3.4 Download graph-node complier,we have modified the graphnode code! Make sure you download it from our github!

sudo apt install -y cmake

cd ${HOME}
git clone https://github.com/gzliudan/graph-node graph-node.xdc
cd graph-node.xdc
git checkout -b xdc origin/xdc
cargo build --release

3.5 Create a startup Script

cat > ${HOME}/start-graph-indexer.sh <<-\EOF
#!/bin/bash

export CHAIN="$1"

# export GRAPH_LOG="trace"
# export GRAPH_LOG="debug"
export GRAPH_LOG_TIME_FORMAT="%Y-%m-%d %H:%M:%S"
LOG_FILE="/var/log/graph/indexer-${CHAIN}-`/usr/bin/date +%Y%m%d`.log"

${HOME}/graph-node.xdc/target/release/graph-node \
  --node-id indexer_${CHAIN} \
  --postgres-url postgresql://graph:daniel2022@localhost:5432/${CHAIN}_db \
  --ethereum-rpc ${CHAIN}:https://arpc.${CHAIN}.network \
  --ipfs 127.0.0.1:5001 \
  --expensive-queries-filename /etc/graph-node/expensive-queries.txt \
  >> ${LOG_FILE} 2>&1
EOF

chmod +x ${HOME}/start-graph-indexer.sh

3.6 Create service files

USER="`whoami`"
sudo tee /usr/lib/systemd/system/graph-indexer@.service <<EOF
[Unit]
Description=Graph indexer node %i
After=network.target ipfs.service
Wants=network.target ipfs.service

[Service]
User=${USER}
Group=${USER}
WorkingDirectory=/home/${USER}/
StandardOutput=journal
StandardError=journal
Type=simple
Restart=always
RestartSec=5
ExecStart=/home/${USER}/start-graph-indexer.sh %i

[Install]
WantedBy=default.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable graph-indexer@apothem
# sudo systemctl disable graph-indexer@apothem
sudo systemctl start graph-indexer@apothem
systemctl status graph-indexer@apothem
# sudo systemctl stop graph-indexer@apothem

3.7 Time setting to restart the indexer service

examine cron job: sudo crontab -l -u root
add corn job to root account: sudo crontab -e -u root

restart service daily: 0 0 * * * /usr/bin/systemctl restart graph-indexer@apothem.service

examine the task: sudo crontab -l -u root

3.8 TODO

you can always pull the source code, comply and restart the service by adding a timed task if you need.

Query with sample

Tutorial： https://thegraph.com/hosted-service/subgraph/gzliudan/bad-token-mumbai

http://103.101.129.136:8000/subgraphs/name/gzliudan/bad-token-subgraph-apothem

{
  erc20Contracts(first: 5) {
    id
    name
    symbol
    decimals
    totalSupply {
      value
      valueExact
    }
  }
  accounts(first: 5) {
    id
    isErc20
    blackLists {
      id
    }
    Erc20balances {
      id
    }
  }
  blackLists(first: 5) {
    id
    members {
      id
    }
  }
}

xdc dosent suport eth_call with specific blockhash

xu zhaolin — Fri, 12 Aug 2022 16:46:05 +0000

We are doing graph-node compatibility project.

We post

curl -X POST https://arpc.apothem.network/ -H "Content-Type: application/json" -d '
{
"id": 348,
"jsonrpc": "2.0",
"method": "eth_call",
"params": [
{
"data": "0x06fdde03",
"gas": "0x2faf080",
"to": "0x53350795c11cee781a7e174479778f848d76ab2a"
},
{
"blockHash": "0xe8ec0bfc021f839fd0a69ae89689e309632af2cc434e8a096732068f0736864b"
}
]
}'

The response became

{"jsonrpc":"2.0","id":348,"error":{"code":-32602,"message":"invalidargument 1: hex string without 0x prefix"}}

It seems like xdc doesn’t support eth_call ?

One more thing worth mentioning here.

If we change the second parameter to “latest “ it will return the correct value

{"jsonrpc":"2.0","id":348,"result":"0x0000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000000f44656d6f20476f6c6420546f6b656e0000000000000000000000000000000000"}

which means xdc only supports eth_call when parameter is “latest”
but not specific blockhash.

This is a serious problem for graph-node compatibility because graph’s subgraph will extract readonly function via eth_call.

issue：arpc responses with a blockhash that doesnt exist.

xu zhaolin — Fri, 05 Aug 2022 09:27:06 +0000

we are currently doing graphnode compatibility project,when we send request to arpc

curl -X POST https://arpc.apothem.network/ -H "Content-Type: application/json" -d '
{
"id": 1610,
"jsonrpc": "2.0",
"method": "eth_getLogs",
"params": [
{
"address": "0x53350795c11cee781a7e174479778f848d76ab2a",
"fromBlock": "0x22b2277",
"toBlock": "0x22b2277",
"topics": [
[
"0x8c5be1e5ebec7d5bd14f71427d1e84f3dd0314c0f7b2291e5b200ac8c7c3b925",
"0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
"0x6a12b3df6cba4203bd7fd06b816789f87de8c594299aed5717ae070fac781bac"
]
]
}
]
}'

it returns the following

{
"id": 1610,
"jsonrpc": "2.0",
"result": [
{
"address": "xdc53350795c11cee781a7e174479778f848d76ab2a",
"blockHash": "0x194a513ea40c94fccaca470cccb24c2ff3bd6398dec9a04e3eb3a4bfef8a330e",
"blockNumber": "0x22b2277",
"data": "0x000000000000000000000000000000000000000000084595161401484a000000",
"logIndex": "0x2",
"removed": false,
"topics": [
"0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
"0x0000000000000000000000000000000000000000000000000000000000000000",
"0x000000000000000000000000ce77393582d39048a4fc73770ab19d2b239c45dd"
],
"transactionHash": "0xbebc0bab1a0f9844b86b8edf3a169bb1eef9337ac8d2684d5ae758b6521c12fe",
"transactionIndex": "0x1"
}
]
}

and we found that the blockHash is 0x194a513ea40c94fccaca470cccb24c2ff3bd6398dec9a04e3eb3a4bfef8a330e which doesnt exist.

Anyone knows the reason?