Quan Tong
How to send an HTTP request without using curl?
2022-05-22
Problem
We are using JWT validation. For some reasons, when testing on staging, we got 401 error:
1[GIN] 2022/05/20 - 14:20:57 | 401 | 2.588128ms | 127.0.0.1 | POST "/v1/endpoint"
Troubleshooting
After looking at the source code, we need to set the operation_debug to true to see what caused that error:
12022/05/20 08:31:26 KRAKEND ERROR: [ENDPOINT: /v1/endpoint][JWTValidator] Unable to validate the token: should have a JSON content type for JWKS endpoint
Troubleshooting slow network file transfer
2021-06-25
We have a Hadoop data cluster. Recently, my colleague noticed that data transfer between servers was slow. Not all are slow; only half of them have problems.
The first tool that comes to mind is iperf3. After installing, I run iperf3 -s then iperf3 -c. While other servers have bandwidth > 900 Mbits/sec, the slow ones are just:
1[ ID] Interval Transfer Bandwidth Retr Cwnd 2[ 4] 0.00-1.00 sec 11.9 MBytes 99.4 Mbits/sec 0 170 KBytes 3[ 4] 1.00-2.00 sec 11.1 MBytes 93.3 Mbits/sec 0 204 KBytes 4[ 4] 2.00-3.00 sec 10.9 MBytes 91.7 Mbits/sec 0 204 KBytes 5[ 4] 3.00-4.00 sec 8.26 MBytes 69.3 Mbits/sec 102 154 KBytes 6[ 4] 4.00-5.00 sec 10.9 MBytes 91.8 Mbits/sec 3 157 KBytes 7[ 4] 5.00-6.00 sec 10.9 MBytes 91.7 Mbits/sec 0 165 KBytes 8[ 4] 6.00-7.00 sec 10.9 MBytes 91.7 Mbits/sec 0 168 KBytes 9[ 4] 7.00-8.00 sec 10.9 MBytes 91.7 Mbits/sec 0 173 KBytes 10[ 4] 8.00-9.00 sec 10.9 MBytes 91.7 Mbits/sec 0 173 KBytes 11[ 4] 9.00-10.00 sec 10.9 MBytes 91.7 Mbits/sec 0 173 KBytes 12- - - - - - - - - - - - - - - - - - - - - - - - - 13[ ID] Interval Transfer Bandwidth Retr 14[ 4] 0.00-10.00 sec 108 MBytes 90.4 Mbits/sec 105 sender 15[ 4] 0.00-10.00 sec 107 MBytes 89.5 Mbits/sec receiver
How a gRPC message uses length-prefixed framing?
2020-05-06
Once we have the encoded data to send to the other end, we need to package the data in a way that other end can easily extract the information. gRPC uses a message framing technique called length-prefixed framing.
Filter gRPC messages:
1$ tshark -r grpc.pcapng -Y 'grpc'
Find out a stream: DATA:
1216 923.033355 127.0.0.1 → 127.0.0.1 GRPC 108 DATA[1] (GRPC) (PROTOBUF)
and show the packet details:
1tshark -r grpc.pcapng -Y "frame.number == 216" -V
How to show all HTTP2 headers using tshark?
2020-05-05
After sniffing with tcpdump, how can I show all HTTP2 header using tshark?
First, find the frame number based on method:
1$ tshark -r grpc.pcapng -Y 'http2.headers.path contains "getBook"' 2214 923.033174 127.0.0.1 → 127.0.0.1 HTTP2 150 HEADERS[1]: POST /book.BookInfo/getBook
and then show the packet details:
1tshark -r grpc.pcapng -Y "frame.number == 214" -V 2 3HyperText Transfer Protocol 2 4 Stream: HEADERS, Stream ID: 1, Length 85, POST /book.BookInfo/getBook 5 Length: 85 6 Type: HEADERS (1) 7 Flags: 0x04 8 .... ...0 = End Stream: False 9 .... .1.. = End Headers: True 10 .... 0... = Padded: False 11 ..0. .... = Priority: False 12 00.0 ..0. = Unused: 0x00 13 0... .... .... .... .... .... .... .... = Reserved: 0x0 14 .000 0000 0000 0000 0000 0000 0000 0001 = Stream Identifier: 1 15 [Pad Length: 0] 16 Header Block Fragment: 8386459162339faaf74e7eb92a94ec4c54dd39faff418b08… 17 [Header Length: 216] 18 [Header Count: 8] 19 Header: :method: POST 20 Name Length: 7 21 Name: :method 22 Value Length: 4 23 Value: POST 24 :method: POST 25 [Unescaped: POST] 26 Representation: Indexed Header Field 27 Index: 3 28 Header: :scheme: http 29 Name Length: 7 30 Name: :scheme 31 Value Length: 4 32 Value: http 33 :scheme: http 34 [Unescaped: http] 35 Representation: Indexed Header Field 36 Index: 6 37 Header: :path: /book.BookInfo/getBook 38 Name Length: 5 39 Name: :path 40 Value Length: 22 41 Value: /book.BookInfo/getBook 42 :path: /book.BookInfo/getBook 43 [Unescaped: /book.BookInfo/getBook] 44 Representation: Literal Header Field with Incremental Indexing - Indexed Name 45 Index: 5 46 Header: :authority: 127.0.0.1:50051 47 Name Length: 10 48 Name: :authority 49 Value Length: 15 50 Value: 127.0.0.1:50051 51 :authority: 127.0.0.1:50051 52 [Unescaped: 127.0.0.1:50051] 53 Representation: Literal Header Field with Incremental Indexing - Indexed Name 54 Index: 1 55 Header: content-type: application/grpc 56 Name Length: 12 57 Name: content-type 58 Value Length: 16 59 Value: application/grpc 60 content-type: application/grpc 61 [Unescaped: application/grpc] 62 Representation: Literal Header Field with Incremental Indexing - Indexed Name 63 Index: 31 64 Header: user-agent: grpc-go/1.24.0 65 Name Length: 10 66 Name: user-agent 67 Value Length: 14 68 Value: grpc-go/1.24.0 69 user-agent: grpc-go/1.24.0 70 [Unescaped: grpc-go/1.24.0] 71 Representation: Literal Header Field with Incremental Indexing - Indexed Name 72 Index: 58 73 Header: te: trailers 74 Name Length: 2 75 Name: te 76 Value Length: 8 77 Value: trailers 78 [Unescaped: trailers] 79 Representation: Literal Header Field with Incremental Indexing - New Name 80 Header: grpc-client: evans 81 Name Length: 11 82 Name: grpc-client 83 Value Length: 5 84 Value: evans 85 [Unescaped: evans] 86 Representation: Literal Header Field with Incremental Indexing - New Name