15

Using a device from mini-circuits that used to work just fine when queried with an older version of curl.

As can be seen here many commands expect a question mark as part of the command. This worked with no issue in the past and I can still run with it in an old container. This is visible with curls -v flag.

With Curl 7.29.0.

$ curl -v 'http://10.13.36.28/:MN?'
* About to connect() to 10.13.36.28 port 80 (#0)
*   Trying 10.13.36.28...
* Connected to 10.13.36.28 (10.13.36.28) port 80 (#0)
> GET /:MN? HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.13.36.28
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: text/html
< Content-Length: 16
< 
* Connection #0 to host 10.13.36.28 left intact
...

With Curl 8.11.1.

$ curl -v 'http://10.13.36.28/:MN?'
* Uses proxy env variable no_proxy == 'localhost,127.0.0.0/8,::1'
*   Trying 10.13.36.28:80...
* Connected to 10.13.36.28 (10.13.36.28) port 80
* using HTTP/1.x
> GET /:MN HTTP/1.1
> Host: 10.13.36.28
> User-Agent: curl/8.11.1
> Accept: */*
> 
* Request completely sent off
< HTTP/1.1 200 OK
< Content-Type: text/html
< Content-Length: 57
< 
* Connection #0 to host 10.13.36.28 left intact
...

The result is that the device does not recognize the command and returns an error.

What I have tried.

  • Use url encoding %3f.
  • Escape common to many command line tools \?.

What I did not try (yet).

  • Looking at the curl code.

My non-curl alternatives.

  • wget -qO - 'http://10.13.36.28/:MN?'.
  • Communicate with the device's telnet port instead.
  • Write in Python.

Not a big fan of these alternatives and wondering if this is a bug or a feature in curl (given that usually ? is an argument separator?)

Addendum

Not that it matters to this question but an undesired side-effect (some will say bug) of this curl change is that using a Copy as cUrl from a browser's DevTools -> Network for such a URL would produce a failing command, below pasted from Chrome

curl 'http://10.13.36.28/:MN?' \
  -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7' \
  -H 'Accept-Language: en-US,en;q=0.9' \
  -H 'Connection: keep-alive' \
  -H 'Upgrade-Insecure-Requests: 1' \
  -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36' \
  -H 'dnt: 1' \
  -H 'sec-gpc: 1' \
  --insecure
Improve this question
0

1 Answer 1

Reset to default
22

You can use the --request-target parameter to explicitly set what path you want to send.

--request-target

(HTTP) Use an alternative target (path) instead of using the path as provided in the URL. Particularly useful when wanting to issue HTTP requests without leading slash or other data that does not follow the regular URL pattern, like "OPTIONS *".

curl passes on the verbatim string you give it in the request without any filter or other safe guards. That includes white space and control characters.

If --request-target is provided several times, the last set value is used.

Example usage:

curl --request-target '/:MN?' 'https://example.com'

I don't know how curl works behind the scenes, but it's possible CURLU_GET_EMPTY which was added to libcurl's curl_url_get is the cause of the change.

CURLU_GET_EMPTY

When this flag is used in curl_url_get(), it makes the function return empty query and fragments parts or when used in the full URL. By default, libcurl otherwise considers empty parts non-existing.

An empty query part is one where this is nothing following the question mark (before the possible fragment). An empty fragments part is one where there is nothing following the hash sign.

(Added in curl 8.8.0)

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.