xu zhaolin

Posted on Aug 16, 2022 • Updated on Sep 1, 2022

Graphnode Xinfin Compatibility Project （Done）

Dear all，

We have finished all the work that needed for graphnode to be compatible with XinFin netwwork. You can duplicate our steps to get the same results!
**
We have modified some of the graph source code to make it compatible with xdc network! you must use the code we provide to implement our method.**

https://github.com/gzliudan/graph-node
Graphnode overview,we have modifeid it to be compatitable with the xdc!!!!

https://github.com/gzliudan/parity-common
gzliudan/parity-common: Collection of crates used in Parity projects

https://github.com/gzliudan/bad-token-subgraph
The subgraph for XRC20 on blockchain XinFin and Apothem

https://github.com/gzliudan/stake-subgraph
The subgraph for StakeContract on blockchain XinFin and Apothem

Step-by-Step tutorial

Deploy the Graph

Deploy three daemons：IPFS, PostgreSql, and graph-indexer. Ipfs and graph-indexer should run with the current user right.

1、Install IPFS

full tutorial： https://docs.ipfs.io/install/command-line/#official-distributions

1.1 Modify UDP Receive Buffer Size

Tutorial： https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size

sudo sysctl -w net.core.rmem_max=2500000
echo "net.core.rmem_max=2500000" | sudo tee -a /etc/sysctl.conf

1.2 Instal IPFS

Tutorial： https://docs.ipfs.io/install/command-line/#official-distributions

wget https://dist.ipfs.io/kubo/v0.14.0/kubo_v0.14.0_linux-amd64.tar.gz
tar -xvzf kubo_v0.14.0_linux-amd64.tar.gz
cd kubo
sudo bash install.sh
ipfs --version

1.3 Initialize IPFS

export IPFS_PATH="/var/lib/ipfs"
echo 'export IPFS_PATH="/var/lib/ipfs"' >> ${HOME}/.bashrc

USER="`whoami`"
sudo mkdir -p /var/lib/ipfs
sudo chown -R ${USER}:${USER} ${IPFS_PATH}
sudo chmod ug+rwx ${IPFS_PATH}

ipfs init
cp -p ${IPFS_PATH}/config ${IPFS_PATH}/config.bak
sed -i "s#/ip4/127.0.0.1/tcp/5001#/ip4/0.0.0.0/tcp/5001#g" ${IPFS_PATH}/config
ipfs config --json API.HTTPHeaders.Access-Control-Allow-Origin '["http://192.168.2.162:5001", "http://127.0.0.1:5001"]'
ipfs config --json API.HTTPHeaders.Access-Control-Allow-Methods '["PUT", "POST"]'

1.4 Creat Service

sudo tee /etc/systemd/system/ipfs.service <<EOF
[Unit]
Description=InterPlanetary File System (IPFS) daemon
Documentation=https://docs.ipfs.tech/
After=network.target
Requires=network.target

[Service]
Type=simple
User=${USER}
Group=${USER}
Environment="IPFS_PATH=${IPFS_PATH}"
Environment="IPFS_FD_MAX=8192"
LimitNOFILE=8192
MemorySwapMax=0
# TimeoutStartSec=infinity
WorkingDirectory=${IPFS_PATH}
ExecStart=/usr/local/bin/ipfs daemon
RestartSec=10
Restart=always
KillSignal=SIGINT
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=default.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable ipfs
sudo systemctl start ipfs
sudo systemctl status ipfs
curl -X POST http://127.0.0.1:5001/api/v0/version
# response: {"Version":"0.14.0","Commit":"","Repo":"12","System":"amd64/linux","Golang":"go1.18.3"}

2、Install PostgreSQL

Tutorial： https://www.postgresql.org/download/linux/ubuntu/

database username: graph
password: daniel2022
listener ports: 5432
database name: apothem_db、xinfin_db

2.1 Modify the kernel parameter

cat /proc/sys/vm/swappiness
sudo bash -c "sysctl -w vm.swappiness=1"
echo "vm.swappiness=1" | sudo tee -a /etc/sysctl.conf
cat /proc/sys/vm/swappiness

2.2 Install language support(i use Chinese here)

sudo apt install -y language-pack-zh*

2.3 Install software

# Create the file repository configuration:
sudo sh -c 'echo "deb [arch=$(dpkg --print-architecture)] http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'

# Import the repository signing key:
# wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo tee /etc/apt/trusted.gpg.d/postgresql.asc

# Update the package lists:
sudo apt update

# Install PostgreSQL 14
sudo apt install -y postgresql-14 postgresql-client-14 postgresql-server-dev-14 libpq-dev

2.4 Backup config file

cd /etc/postgresql/14/main
sudo cp -p pg_hba.conf pg_hba.conf.`date +%Y%m%d-%H%M%S`
sudo cp -p postgresql.conf postgresql.conf.`date +%Y%m%d-%H%M%S`

2.5 Modify pg_hba.conf

sudo tee /etc/postgresql/14/main/pg_hba.conf <<-\EOF
# Database administrative login by Unix domain socket
local   all             postgres                                peer

# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     md5
# IPv4 local connections:
host    all             all             127.0.0.1/32            md5
# IPv6 local connections:
host    all             all             ::1/128                 md5
# Allow replication connections from localhost, by a user with the
# replication privilege.
local   replication     all                                     peer
host    replication     all             127.0.0.1/32            md5
host    replication     all             ::1/128                 md5
EOF

2.6 Some parameters adjustments

sudo tee /etc/postgresql/14/main/conf.d/liudan.conf <<-\EOF
listen_addresses = '*'
port = 5432

tcp_keepalives_idle = 300
tcp_keepalives_interval = 20
tcp_keepalives_count = 9

log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log'
log_line_prefix = '%m:%r:%u@%d:[%p]: '
log_rotation_age = 1d
lc_messages = 'en_US.UTF-8'            # log is shown in English

shared_preload_libraries = 'pg_stat_statements'
EOF

2.7 change os user's postgres password

echo postgres:daniel2022 | sudo chpasswd

2.8 start the database

# pg_ctlcluster 14 main start
sudo systemctl daemon-reload
sudo systemctl restart postgresql@14-main
systemctl status postgresql@14-main

2.9 change database user's postgres password

su - postgres
psql -c "ALTER ROLE postgres WITH PASSWORD 'daniel2022';"
exit

2.10 Creat Database

create user of the database: graph
create database： apothem_db and xinfin_db

PGPASSWORD=daniel2022 psql -X -h 127.0.0.1 -U postgres postgres <<-\EOF
CREATE USER graph WITH PASSWORD 'daniel2022' nocreatedb;
CREATE DATABASE apothem_db WITH OWNER=graph TEMPLATE=template0 LC_COLLATE='zh_CN.UTF8' LC_CTYPE='zh_CN.UTF8' ENCODING='utf8';
CREATE DATABASE xinfin_db WITH OWNER=graph TEMPLATE=template0 LC_COLLATE='zh_CN.UTF8' LC_CTYPE='zh_CN.UTF8' ENCODING='utf8';
EOF

2.11 Create extension

create extension： pg_trgm、pg_stat_statements、btree_gist、postgres_fdw

2.11.1 testnet apothem

PGPASSWORD=daniel2022 psql -X -h 127.0.0.1 -U postgres apothem_db <<-\EOF
CREATE EXTENSION pg_trgm;
CREATE EXTENSION pg_stat_statements;
GRANT ALL ON FUNCTION pg_stat_statements_reset TO graph;
CREATE EXTENSION btree_gist;
CREATE EXTENSION postgres_fdw;
GRANT USAGE ON FOREIGN DATA WRAPPER postgres_fdw TO graph;
EOF

2.11.2 production net xinfin

PGPASSWORD=daniel2022 psql -X -h 127.0.0.1 -U postgres xinfin_db <<-\EOF
CREATE EXTENSION pg_trgm;
CREATE EXTENSION pg_stat_statements;
GRANT ALL ON FUNCTION pg_stat_statements_reset TO graph;
CREATE EXTENSION btree_gist;
CREATE EXTENSION postgres_fdw;
GRANT USAGE ON FOREIGN DATA WRAPPER postgres_fdw TO graph;
EOF

2.12 setting up environment for psql ：

cat >> ${HOME}/.bashrc <<-\EOF
export PGHOST=127.0.0.1
export PGPORT=5432
export PGDATABASE=apothem_db
export PGUSER=graph
export PGPASSWORD=daniel2022
EOF

source ${HOME}/.bashrc

cat > ${HOME}/.psqlrc <<-\EOF
\set PROMPT1 '%`date +%H:%M:%S` (%n@%M:%>)%/%R%#%x '
\set PROMPT2 '%M %n@%/%R%# '
\timing on
EOF

3、Install graph-node

Tutorial：

https://github.com/graphprotocol/graph-node/blob/master/docs/config.md
https://github.com/graphprotocol/graph-node/blob/master/node/resources/tests/full_config.toml
Running a Graph Node on Moonbeam: https://docs.moonbeam.network/node-operators/indexer-nodes/thegraph-node/
Deploy and Configure Graph-node: https://docs.thegraph.academy/official-docs/indexer/testnet/graph-protocol-testnet-baremetal/3_deployandconfiguregraphnode

3.1 Create log files directory

USER="`whoami`"
sudo mkdir -p /var/log/graph
sudo chown -R ${USER}:${USER} /var/log/graph

3.2 Create config files directory

USER="`whoami`"
sudo mkdir -p /etc/graph-node
sudo chown -R ${USER}:${USER} /etc/graph-node
touch /etc/graph-node/expensive-queries.txt

3.3 Install Rust

安装或者更新:

cargo -V
if [ "$?" == 0 ] ; then
    rustup update stable
else
    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
    source ${HOME}/.cargo/env
fi
cargo -V

3.4 Download graph-node complier,we have modified the graphnode code! Make sure you download it from our github!

sudo apt install -y cmake

cd ${HOME}
git clone https://github.com/gzliudan/graph-node graph-node.xdc
cd graph-node.xdc
git checkout -b xdc origin/xdc
cargo build --release

3.5 Create a startup Script

cat > ${HOME}/start-graph-indexer.sh <<-\EOF
#!/bin/bash

export CHAIN="$1"

# export GRAPH_LOG="trace"
# export GRAPH_LOG="debug"
export GRAPH_LOG_TIME_FORMAT="%Y-%m-%d %H:%M:%S"
LOG_FILE="/var/log/graph/indexer-${CHAIN}-`/usr/bin/date +%Y%m%d`.log"

${HOME}/graph-node.xdc/target/release/graph-node \
  --node-id indexer_${CHAIN} \
  --postgres-url postgresql://graph:daniel2022@localhost:5432/${CHAIN}_db \
  --ethereum-rpc ${CHAIN}:https://arpc.${CHAIN}.network \
  --ipfs 127.0.0.1:5001 \
  --expensive-queries-filename /etc/graph-node/expensive-queries.txt \
  >> ${LOG_FILE} 2>&1
EOF

chmod +x ${HOME}/start-graph-indexer.sh

3.6 Create service files

USER="`whoami`"
sudo tee /usr/lib/systemd/system/graph-indexer@.service <<EOF
[Unit]
Description=Graph indexer node %i
After=network.target ipfs.service
Wants=network.target ipfs.service

[Service]
User=${USER}
Group=${USER}
WorkingDirectory=/home/${USER}/
StandardOutput=journal
StandardError=journal
Type=simple
Restart=always
RestartSec=5
ExecStart=/home/${USER}/start-graph-indexer.sh %i

[Install]
WantedBy=default.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable graph-indexer@apothem
# sudo systemctl disable graph-indexer@apothem
sudo systemctl start graph-indexer@apothem
systemctl status graph-indexer@apothem
# sudo systemctl stop graph-indexer@apothem

3.7 Time setting to restart the indexer service

examine cron job: sudo crontab -l -u root
add corn job to root account: sudo crontab -e -u root

restart service daily: 0 0 * * * /usr/bin/systemctl restart graph-indexer@apothem.service

examine the task: sudo crontab -l -u root

3.8 TODO

you can always pull the source code, comply and restart the service by adding a timed task if you need.

Query with sample

Tutorial： https://thegraph.com/hosted-service/subgraph/gzliudan/bad-token-mumbai

http://103.101.129.136:8000/subgraphs/name/gzliudan/bad-token-subgraph-apothem

{
  erc20Contracts(first: 5) {
    id
    name
    symbol
    decimals
    totalSupply {
      value
      valueExact
    }
  }
  accounts(first: 5) {
    id
    isErc20
    blackLists {
      id
    }
    Erc20balances {
      id
    }
  }
  blackLists(first: 5) {
    id
    members {
      id
    }
  }
}

Discussion (20)

ncode • Aug 16 '22

Can we do this using docker?
like with a simple docker-compose up

Mr. Popovich • Aug 16 '22

totally agree. by making it with docker-compose we will have only one config file and only one line of code to run it

Salomon Morales • Aug 17 '22

I agree with the approach of Docker but we have to keep in mind that if Docker version gets updated, to ensure that whatever it's deploy did not get affected by the vers update. That's my only concern but I agree that we should simplify what we can and make it easier (ready-to-deploy).

Mr. Popovich • Aug 22 '22

Maybe we talk about different things. What im saying is to make one docker-compose file that will have all scripts. No need to deploy image to docker-hub.

Jon McBee • Aug 17 '22

I am in favor in taking whatever approach simplifies usage of the tool for the developer community as it will not only make this easier to consume but easier to support when issues arise.

Daniel Liu • Feb 4 '23

Support docker now:
github.com/Carry-So/graph-node#1-d...

Mr. Popovich • Feb 4 '23

how can we increase indexing process?

Eduardo Robles • Aug 30 '22

I've been trying to implement a subgraph using graph-protocol's graph-node by changing this file to have ethereum: 'mainnet:https://rpc.apothem.network'.

But running docker-compose up with this gives the following error.
Aug 29 20:46:40.700 WARN Trying again after eth_getBlockByNumber(0, false) RPC call failed (attempt #11) with result Err(Decoder error: Error("invalid length 43, expected a (both 0x-prefixed or not) hex string with length of 40", line: 0, column: 0)), provider: mainnet-rpc-0

I was kindly pointed to this post which does say that it is possible to have a subgraph, but haven't been able to tell the difference from my code to the one in the examples.

Can someone point me in the right direction here? What am i missing?

xu zhaolin • Aug 30 '22

you need to instal from the adress we provide

xu zhaolin • Aug 30 '22

looking into it

xu zhaolin • Aug 30 '22

Hi
You need to use the GitHub we provide we have modified the graph source code to be adoptable with xdc network.

Eduardo Robles • Aug 31 '22

I have cloned gzliudan/graph-node but still get these errors | Aug 31 13:12:23.552 WARN Trying again after eth_getBlockByNumber(0, false) RPC call failed (attempt #16) with result Err(Decoder error: Error("invalid length 43, expected a (both 0x-prefixed or not) hex string with length of 40", line: 0, column: 0)), provider: mainnet-rpc-0 and when trying to deploy Failed to deploy to Graph node http://localhost:8020/: network not supported by registrar: no network mainnet found on chain ethereum

xu zhaolin • Sep 5 '22

let me check

xu zhaolin • Sep 5 '22

you havent switched to the xdc branch

git checkout -b xdc origin/xdc

Belle • Aug 17 '22

As what Mr. Popovich has said, it will be good to have a ready made product that everyone can use through docker-compose. The solution provided is great but then in order for it to work, you have make it from scratch by going through all the steps and there is no guarantee of its outcome. The room for error is huge compared to having it in docker-compose.

ncode • Dec 7 '22 • Edited on Dec 7

Can you explain the usecase of --expensive-quries-filename? it says unnecessary parameter while deploying

Daniel Liu • Dec 8 '22

The name should be expensive-queries-filename, not expensive-quries-filename.

ncode • Dec 8 '22

Can you just explain the use case of that ? Please check the screenshot I haven't made any spelling mistakes🤦‍♂️🤦‍♂️

xu zhaolin • Aug 18 '22 • Edited on Aug 18

Technically speaking, it can be done with docker. But the current version is unstable,we dont suggest to use docker. In addition, in the production enviroment, ipfs and pg database is privately deployed in a cluster manner rather than putting them in docker-compose.

Daniel Liu • Oct 19 '22

The latest version is here: github.com/Carry-So/graph-node

Developers Forum for XinFin XDC Network

Developers Forum for XinFin XDC Network is a community of 3,490 amazing users

Graphnode Xinfin Compatibility Project （Done）

Deploy the Graph

1、Install IPFS

1.1 Modify UDP Receive Buffer Size

1.2 Instal IPFS

1.3 Initialize IPFS

1.4 Creat Service

2、Install PostgreSQL

2.1 Modify the kernel parameter

2.2 Install language support(i use Chinese here)

2.3 Install software

2.4 Backup config file

2.5 Modify pg_hba.conf

2.6 Some parameters adjustments

2.7 change os user's postgres password

2.8 start the database

2.9 change database user's postgres password

2.10 Creat Database

2.11 Create extension

2.11.1 testnet apothem

2.11.2 production net xinfin

2.12 setting up environment for psql ：

3、Install graph-node

3.1 Create log files directory

3.2 Create config files directory

3.3 Install Rust

3.4 Download graph-node complier,we have modified the graphnode code! Make sure you download it from our github!

3.5 Create a startup Script

3.6 Create service files

3.7 Time setting to restart the indexer service

3.8 TODO

Query with sample

Discussion (20)

Read next

Subnet Troubleshooting Support Group

What Is XDC Crypto?

When I download the snapshot and start the masternode it won't connect to network and download blocks

DOPU Contract Migration: A New Chapter for Our Community