links
371 shaares
3 results
tagged
curl
Curl passing two Query string or arguments:
The -G argument will append the data specified in -d and --data-urlencode arguments at the end of the request URL, joining all data pieces with the & character and separating them from the URL with the ? character.
Example:
curl -vv -G -d "track=randwick" -d "race=1" https://s9iwqktpp8.execute-api.ap-southeast-2.amazonaws.com/puntwit
> GET /puntwit?track=randwick&race=1 HTTP/2
> Host: s9iwqktpp8.execute-api.ap-southeast-2.amazonaws.com
> User-Agent: curl/7.58.0
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200
< date: Tue, 08 Jun 2021 03:18:54 GMT
< content-type: application/json
< content-length: 1017
< apigw-requestid: AlgwyjqBywMEMSw=
<
* Connection #0 to host s9iwqktpp8.execute-api.ap-southeast-2.amazonaws.com left intact
{"version":"2.0","routeKey":"GET /puntwit","rawPath":"/puntwit","rawQueryString":"track=randwick&race=1","headers":{"accept":"*/*","content-length":"0","host":"s9iwqktpp8.execute-api.ap-southeast-2.amazonaws.com","user-agent":"curl/7.58.0","x-amzn-trace-id":"Root=1-60b9d48e-3c1555b220ef70ce1f7d2933","x-forwarded-for":"13.54.150.130","x-forwarded-port":"443","x-forwarded-proto":"https"},"queryStringParameters":{"race":"1","track":"randwick"},"requestContext":{"accountId":"302766791300","apiId":"s9iwqktpp8","domainName":"s9iwqktpp8.execute-api.ap-southeast-2.amazonaws.com","domainPrefix":"s9iwqktpp8","http":{"method":"GET","path":"/puntwit","protocol":"HTTP/1.1","sourceIp":"13.54.150.130","userAgent":"curl/7.58.0"},"requestId":"AY4mSj-ASwMENlA=","routeKey":"GET /puntwit","stage":"$default","time":"04/Jun/2021:07:21:50 +0000","timeEpoch":1622791310453},"isBase64Encoded":false}
The -G argument will append the data specified in -d and --data-urlencode arguments at the end of the request URL, joining all data pieces with the & character and separating them from the URL with the ? character.
Example:
curl -vv -G -d "track=randwick" -d "race=1" https://s9iwqktpp8.execute-api.ap-southeast-2.amazonaws.com/puntwit
> GET /puntwit?track=randwick&race=1 HTTP/2
> Host: s9iwqktpp8.execute-api.ap-southeast-2.amazonaws.com
> User-Agent: curl/7.58.0
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200
< date: Tue, 08 Jun 2021 03:18:54 GMT
< content-type: application/json
< content-length: 1017
< apigw-requestid: AlgwyjqBywMEMSw=
<
* Connection #0 to host s9iwqktpp8.execute-api.ap-southeast-2.amazonaws.com left intact
{"version":"2.0","routeKey":"GET /puntwit","rawPath":"/puntwit","rawQueryString":"track=randwick&race=1","headers":{"accept":"*/*","content-length":"0","host":"s9iwqktpp8.execute-api.ap-southeast-2.amazonaws.com","user-agent":"curl/7.58.0","x-amzn-trace-id":"Root=1-60b9d48e-3c1555b220ef70ce1f7d2933","x-forwarded-for":"13.54.150.130","x-forwarded-port":"443","x-forwarded-proto":"https"},"queryStringParameters":{"race":"1","track":"randwick"},"requestContext":{"accountId":"302766791300","apiId":"s9iwqktpp8","domainName":"s9iwqktpp8.execute-api.ap-southeast-2.amazonaws.com","domainPrefix":"s9iwqktpp8","http":{"method":"GET","path":"/puntwit","protocol":"HTTP/1.1","sourceIp":"13.54.150.130","userAgent":"curl/7.58.0"},"requestId":"AY4mSj-ASwMENlA=","routeKey":"GET /puntwit","stage":"$default","time":"04/Jun/2021:07:21:50 +0000","timeEpoch":1622791310453},"isBase64Encoded":false}
Good breakdown of printing timing breakdown of curl. For example command:
curl -L --output /dev/null --silent --show-error --write-out 'lookup: %{time_namelookup}\nconnect: %{time_connect}\nappconnect: %{time_appconnect}\npretransfer: %{time_pretransfer}\nredirect: %{time_redirect}\nstarttransfer: %{time_starttransfer}\ntotal: %{time_total}\n' 'https://google.com'
lookup: 0.038
connect: 0.063
appconnect: 0.177
pretransfer: 0.178
redirect: 0.225
starttransfer: 0.679
total: 0.929
From the man pages -
lookup: The time, in seconds, it took from the start until the name resolving was completed.
connect: The time, in seconds, it took from the start until the TCP connect to the remote host (or proxy) was completed.
appconnect: The time, in seconds, it took from the start until the SSL/SSH/etc connect/handshake to the remote host was completed. (Added in 7.19.0)
pretransfer: The time, in seconds, it took from the start until the file transfer was just about to begin. This includes all pre-transfer commands and negotiations that are specific to the particular protocol(s) involved.
redirect: The time, in seconds, it took for all redirection steps include name lookup, connect, pretransfer and transfer before the final transaction was started. time_redirect shows the complete execution time for multiple redirections. (Added in 7.12.3)
starttransfer: The time, in seconds, it took from the start until the first byte was just about to be transferred. This includes time_pretransfer and also the time the server needed to calculate the result.
total: The total time, in seconds, that the full operation lasted. The time will be displayed with millisecond resolution.
curl -L --output /dev/null --silent --show-error --write-out 'lookup: %{time_namelookup}\nconnect: %{time_connect}\nappconnect: %{time_appconnect}\npretransfer: %{time_pretransfer}\nredirect: %{time_redirect}\nstarttransfer: %{time_starttransfer}\ntotal: %{time_total}\n' 'https://google.com'
lookup: 0.038
connect: 0.063
appconnect: 0.177
pretransfer: 0.178
redirect: 0.225
starttransfer: 0.679
total: 0.929
From the man pages -
lookup: The time, in seconds, it took from the start until the name resolving was completed.
connect: The time, in seconds, it took from the start until the TCP connect to the remote host (or proxy) was completed.
appconnect: The time, in seconds, it took from the start until the SSL/SSH/etc connect/handshake to the remote host was completed. (Added in 7.19.0)
pretransfer: The time, in seconds, it took from the start until the file transfer was just about to begin. This includes all pre-transfer commands and negotiations that are specific to the particular protocol(s) involved.
redirect: The time, in seconds, it took for all redirection steps include name lookup, connect, pretransfer and transfer before the final transaction was started. time_redirect shows the complete execution time for multiple redirections. (Added in 7.12.3)
starttransfer: The time, in seconds, it took from the start until the first byte was just about to be transferred. This includes time_pretransfer and also the time the server needed to calculate the result.
total: The total time, in seconds, that the full operation lasted. The time will be displayed with millisecond resolution.
How to set manually set tcp keepalives in curl rather than using Linux Kernel default tcp keepalive - 2 hours (7200 seconds)
Linux Kernel Default Values Ref: https://tldp.org/HOWTO/TCP-Keepalive-HOWTO/usingkeepalive.html
----------------------------------------------------------------------------------------------------------------------------------------------
ubuntu@ip-10-0-135-84:~$ sudo cat /proc/sys/net/ipv4/tcp_keepalive_time
7200
ubuntu@ip-10-0-135-84:~$ sudo cat /proc/sys/net/ipv4/tcp_keepalive_intvl
75
ubuntu@ip-10-0-135-84:~$ sudo cat /proc/sys/net/ipv4/tcp_keepalive_probes
9
--keepalive-time <seconds>
This option sets the time a connection needs to remain idle before sending keepalive probes and the time between individual
keepalive probes. It is currently effective on operating systems offering the TCP_KEEPIDLE and TCP_KEEPINTVL socket options (mean‐
ing Linux, recent AIX, HP-UX and more). This option has no effect if --no-keepalive is used.
If this option is used several times, the last one will be used. If unspecified, the option defaults to 60 seconds.
Added in 7.18.0
Linux Kernel Default Values Ref: https://tldp.org/HOWTO/TCP-Keepalive-HOWTO/usingkeepalive.html
----------------------------------------------------------------------------------------------------------------------------------------------
ubuntu@ip-10-0-135-84:~$ sudo cat /proc/sys/net/ipv4/tcp_keepalive_time
7200
ubuntu@ip-10-0-135-84:~$ sudo cat /proc/sys/net/ipv4/tcp_keepalive_intvl
75
ubuntu@ip-10-0-135-84:~$ sudo cat /proc/sys/net/ipv4/tcp_keepalive_probes
9
--keepalive-time <seconds>
This option sets the time a connection needs to remain idle before sending keepalive probes and the time between individual
keepalive probes. It is currently effective on operating systems offering the TCP_KEEPIDLE and TCP_KEEPINTVL socket options (mean‐
ing Linux, recent AIX, HP-UX and more). This option has no effect if --no-keepalive is used.
If this option is used several times, the last one will be used. If unspecified, the option defaults to 60 seconds.
Added in 7.18.0