Streams don't work #13

New Issue

tastytea · 2020-01-03T05:35:14+01:00

tastytea commented

2020-01-03 05:35:14 +01:00

Version or commit: 0.111.5

Streams don't work because the HTTP response is copied into answer in

		src/http.cpp
		Line 243 in 76e12d7093
	
				        StreamCopier::copyToString(body_stream, answer);

after the request is done. With streams the request is never done, so nothing is ever put into answer.

Version or commit: 0.111.5 Streams don't work because the HTTP response is copied into `answer` in https://schlomp.space/tastytea/mastodon-cpp/src/commit/76e12d7093629fcb15575c82bd37b0c7f8961dec/src/http.cpp#L243 after the request is done. With streams the request is never done, so nothing is ever put into `answer`.

tastytea self-assigned this 2020-01-03 05:35:14 +01:00

tastytea added the

bug

label 2020-01-03 05:35:14 +01:00

FSajadi commented

2020-01-04 13:30:10 +01:00

@tastyea Hi, Do you have a schedule to solve this bug? I'm using it.

tastytea commented

2020-01-04 17:27:23 +01:00

As I wrote in the e-mail, this project has become too complicated for me to maintain. I made a series of errors which resulted in too much bloat and a hard to follow code structure. In addition, the HTTP library I chose (POCO) doesn't seem to be suited for streams.

Looking at the code, I've found no immediate way to solve this but found other bugs in my streams implementation.

Fixing this properly would involve a major refactoring, streams would have to be newly implemented and maybe even the HTTP library would have to be replaced. I'm not going to do that, sorry.

As I wrote in the e-mail, this project has become too complicated for me to maintain. I made a series of errors which resulted in too much bloat and a hard to follow code structure. In addition, the HTTP library I chose (POCO) doesn't seem to be suited for streams. Looking at the code, I've found no immediate way to solve this but found other bugs in my streams implementation. Fixing this properly would involve a major refactoring, streams would have to be newly implemented and maybe even the HTTP library would have to be replaced. I'm not going to do that, sorry.

tastytea removed their assignment 2020-01-04 17:27:29 +01:00

FSajadi commented

2020-01-05 15:16:19 +01:00

Sorry, I didn't see your email. Maybe it helps, mastodon-cpp sends notifications(stream) after receiving 10 notifications, in addition, I could work with the stream in the previous version.

Sorry, I didn't see your email. Maybe it helps, `mastodon-cpp` sends notifications(stream) after receiving 10 notifications, in addition, I could work with the stream in the previous version.

tastytea commented

2020-01-05 17:02:29 +01:00

Was the previous version you used before 0.110.0? Because that's when I switched to POCO for HTTP requests.

tastytea commented

2020-01-05 17:18:26 +01:00

You could downgrade to 0.106.0. Streams should work fine in that version. The only downside as far as I can see is that you wouldn't have mastodon-cppConfig.cmake.

You could downgrade to 0.106.0. Streams should work fine in that version. The only downside as far as I can see is that you wouldn't have `mastodon-cppConfig.cmake`.

FSajadi commented

2020-01-06 11:31:06 +01:00

@tastytea Thanks for your quick reply.

I'm not sure which version I used. Any way I downgraded to 0.106.0 and for now get another error at startup in get method. This is the response of the masto.get():

error code :192 server certificate verification failed. CAfile: none CRLfile: none

Do you have any idea?

@tastytea Thanks for your quick reply. I'm not sure which version I used. Any way I downgraded to `0.106.0` and for now get another error at startup in `get` method. This is the response of the `masto.get()`: ``` error code :192 server certificate verification failed. CAfile: none CRLfile: none ``` Do you have any idea?

tastytea commented

2020-01-06 11:51:29 +01:00

The error means that the remote TLS certificate couldn't be verified. Which instance are you using? Do you get the same error with mastodon.social? Do you get an error if you do run curl -I https://<your instance>/api/v1/instance from a terminal?

I just tested example02_stream from 0.106.0 and it works with mastodon.social.

The error means that the remote TLS certificate couldn't be verified. Which instance are you using? Do you get the same error with mastodon.social? Do you get an error if you do run `curl -I https://<your instance>/api/v1/instance` from a terminal? I just tested `example02_stream` from `0.106.0` and it works with mastodon.social.

FSajadi commented

2020-01-06 12:02:57 +01:00

@tastytea I can run the curl -I https://pantherx.social/api/v1/instance without error:

HTTP/2 200 
date: Mon, 06 Jan 2020 11:02:35 GMT
content-type: application/json; charset=utf-8
vary: Accept-Encoding
server: Mastodon
x-frame-options: DENY
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
x-ratelimit-limit: 300
x-ratelimit-remaining: 299
x-ratelimit-reset: 2020-01-06T11:05:00.131841Z
cache-control: max-age=180, public
vary: Accept-Encoding, Origin
etag: W/"8edcecf1a0d165f1805acdf036909129"
x-request-id: c687cad1-657a-4b19-878b-8acfb7216c9e
x-runtime: 0.076069
x-cached: HIT
strict-transport-security: max-age=31536000

@tastytea I can run the `curl -I https://pantherx.social/api/v1/instance` without error: ``` HTTP/2 200 date: Mon, 06 Jan 2020 11:02:35 GMT content-type: application/json; charset=utf-8 vary: Accept-Encoding server: Mastodon x-frame-options: DENY x-content-type-options: nosniff x-xss-protection: 1; mode=block x-ratelimit-limit: 300 x-ratelimit-remaining: 299 x-ratelimit-reset: 2020-01-06T11:05:00.131841Z cache-control: max-age=180, public vary: Accept-Encoding, Origin etag: W/"8edcecf1a0d165f1805acdf036909129" x-request-id: c687cad1-657a-4b19-878b-8acfb7216c9e x-runtime: 0.076069 x-cached: HIT strict-transport-security: max-age=31536000 ```

tastytea commented

2020-01-06 13:15:00 +01:00

That's weird. I can get() to pantherx.social just fine. Which OS are you using? Can you give me the output of ldd /usr/lib*/libcurl.so?

That's weird. I can `get()` to pantherx.social just fine. Which OS are you using? Can you give me the output of `ldd /usr/lib*/libcurl.so`?

tastytea commented

2020-01-06 13:23:15 +01:00

You can also check if the file returned by curl-config --ca exists and if it contains certificates. On my system it is around 200KiB big and was installed by the package ca-certificates.

You can also check if the file returned by `curl-config --ca` exists and if it contains certificates. On my system it is around 200KiB big and was installed by the package `ca-certificates`.

FSajadi commented

2020-01-06 14:56:30 +01:00

@tastytea this is the output of ldd libcurl.so:

linux-vdso.so.1 (0x00007ffc8c8b2000)
        libnghttp2.so.14 => /gnu/store/zavdh2z5mwkakjf1v98x43w1hzjzxkhl-nghttp2-1.39.1-lib/lib/libnghttp2.so.14 (0x00007feac4fee000)
        libidn2.so.0 => /gnu/store/7gabmw9siqrz79slpi1f8i90v3w1638x-libidn2-2.2.0/lib/libidn2.so.0 (0x00007feac4fcf000)
        libnettle.so.7 => /gnu/store/lvnybsygfd6gya6xbdv48g72lb0iqqzx-nettle-3.5.1/lib/libnettle.so.7 (0x00007feac4f92000)
        libgnutls.so.30 => /gnu/store/hfvz18igm68p5yz7z4asn6ph363blp1z-gnutls-3.6.9/lib/libgnutls.so.30 (0x00007feac4df1000)
        libgssapi_krb5.so.2 => /gnu/store/4rs159kgsa0l1svi5vbvn86in7z28bpl-mit-krb5-1.17/lib/libgssapi_krb5.so.2 (0x00007feac4da5000)
        libkrb5.so.3 => /gnu/store/4rs159kgsa0l1svi5vbvn86in7z28bpl-mit-krb5-1.17/lib/libkrb5.so.3 (0x00007feac4cd2000)
        libk5crypto.so.3 => /gnu/store/4rs159kgsa0l1svi5vbvn86in7z28bpl-mit-krb5-1.17/lib/libk5crypto.so.3 (0x00007feac4c9f000)
        libcom_err.so.3 => /gnu/store/4rs159kgsa0l1svi5vbvn86in7z28bpl-mit-krb5-1.17/lib/libcom_err.so.3 (0x00007feac4c9a000)
        libldap-2.4.so.2 => /gnu/store/zb84bam450bpr90kqy43i76cvqhlp0lh-openldap-2.4.47/lib/libldap-2.4.so.2 (0x00007feac4c50000)
        liblber-2.4.so.2 => /gnu/store/zb84bam450bpr90kqy43i76cvqhlp0lh-openldap-2.4.47/lib/liblber-2.4.so.2 (0x00007feac4c3e000)
        libz.so.1 => /gnu/store/qx7p7hiq90mi7r78hcr9cyskccy2j4bg-zlib-1.2.11/lib/libz.so.1 (0x00007feac4c1e000)
        libgcc_s.so.1 => /gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib/lib/libgcc_s.so.1 (0x00007feac4c05000)
        libpthread.so.0 => /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libpthread.so.0 (0x00007feac4be4000)
        libc.so.6 => /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libc.so.6 (0x00007feac4a2a000)
        libunistring.so.2 => /gnu/store/nsikjxykcaqa0zjpfmkqd569bngbv5nl-libunistring-0.9.10/lib/libunistring.so.2 (0x00007feac48a4000)
        libtasn1.so.6 => /gnu/store/lwz8fygpmmsw6h8vrllr56p7ssi5qx33-libtasn1-4.14/lib/libtasn1.so.6 (0x00007feac488f000)
        libhogweed.so.5 => /gnu/store/lvnybsygfd6gya6xbdv48g72lb0iqqzx-nettle-3.5.1/lib/libhogweed.so.5 (0x00007feac4856000)
        libgmp.so.10 => /gnu/store/y7qk8raalgvdnxcglvxa320cfxrjk1x6-gmp-6.1.2/lib/libgmp.so.10 (0x00007feac47c3000)
        /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/ld-linux-x86-64.so.2 (0x00007feac509a000)
        libkrb5support.so.0 => /gnu/store/4rs159kgsa0l1svi5vbvn86in7z28bpl-mit-krb5-1.17/lib/libkrb5support.so.0 (0x00007feac47b2000)
        libdl.so.2 => /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libdl.so.2 (0x00007feac47ad000)
        libresolv.so.2 => /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libresolv.so.2 (0x00007feac4794000)
        libsasl2.so.3 => /gnu/store/06bhhsw843bz70ml9n18adsk13qb3962-cyrus-sasl-2.1.27/lib/libsasl2.so.3 (0x00007feac4776000)

@tastytea this is the output of `ldd libcurl.so`: ``` linux-vdso.so.1 (0x00007ffc8c8b2000) libnghttp2.so.14 => /gnu/store/zavdh2z5mwkakjf1v98x43w1hzjzxkhl-nghttp2-1.39.1-lib/lib/libnghttp2.so.14 (0x00007feac4fee000) libidn2.so.0 => /gnu/store/7gabmw9siqrz79slpi1f8i90v3w1638x-libidn2-2.2.0/lib/libidn2.so.0 (0x00007feac4fcf000) libnettle.so.7 => /gnu/store/lvnybsygfd6gya6xbdv48g72lb0iqqzx-nettle-3.5.1/lib/libnettle.so.7 (0x00007feac4f92000) libgnutls.so.30 => /gnu/store/hfvz18igm68p5yz7z4asn6ph363blp1z-gnutls-3.6.9/lib/libgnutls.so.30 (0x00007feac4df1000) libgssapi_krb5.so.2 => /gnu/store/4rs159kgsa0l1svi5vbvn86in7z28bpl-mit-krb5-1.17/lib/libgssapi_krb5.so.2 (0x00007feac4da5000) libkrb5.so.3 => /gnu/store/4rs159kgsa0l1svi5vbvn86in7z28bpl-mit-krb5-1.17/lib/libkrb5.so.3 (0x00007feac4cd2000) libk5crypto.so.3 => /gnu/store/4rs159kgsa0l1svi5vbvn86in7z28bpl-mit-krb5-1.17/lib/libk5crypto.so.3 (0x00007feac4c9f000) libcom_err.so.3 => /gnu/store/4rs159kgsa0l1svi5vbvn86in7z28bpl-mit-krb5-1.17/lib/libcom_err.so.3 (0x00007feac4c9a000) libldap-2.4.so.2 => /gnu/store/zb84bam450bpr90kqy43i76cvqhlp0lh-openldap-2.4.47/lib/libldap-2.4.so.2 (0x00007feac4c50000) liblber-2.4.so.2 => /gnu/store/zb84bam450bpr90kqy43i76cvqhlp0lh-openldap-2.4.47/lib/liblber-2.4.so.2 (0x00007feac4c3e000) libz.so.1 => /gnu/store/qx7p7hiq90mi7r78hcr9cyskccy2j4bg-zlib-1.2.11/lib/libz.so.1 (0x00007feac4c1e000) libgcc_s.so.1 => /gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib/lib/libgcc_s.so.1 (0x00007feac4c05000) libpthread.so.0 => /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libpthread.so.0 (0x00007feac4be4000) libc.so.6 => /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libc.so.6 (0x00007feac4a2a000) libunistring.so.2 => /gnu/store/nsikjxykcaqa0zjpfmkqd569bngbv5nl-libunistring-0.9.10/lib/libunistring.so.2 (0x00007feac48a4000) libtasn1.so.6 => /gnu/store/lwz8fygpmmsw6h8vrllr56p7ssi5qx33-libtasn1-4.14/lib/libtasn1.so.6 (0x00007feac488f000) libhogweed.so.5 => /gnu/store/lvnybsygfd6gya6xbdv48g72lb0iqqzx-nettle-3.5.1/lib/libhogweed.so.5 (0x00007feac4856000) libgmp.so.10 => /gnu/store/y7qk8raalgvdnxcglvxa320cfxrjk1x6-gmp-6.1.2/lib/libgmp.so.10 (0x00007feac47c3000) /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/ld-linux-x86-64.so.2 (0x00007feac509a000) libkrb5support.so.0 => /gnu/store/4rs159kgsa0l1svi5vbvn86in7z28bpl-mit-krb5-1.17/lib/libkrb5support.so.0 (0x00007feac47b2000) libdl.so.2 => /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libdl.so.2 (0x00007feac47ad000) libresolv.so.2 => /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libresolv.so.2 (0x00007feac4794000) libsasl2.so.3 => /gnu/store/06bhhsw843bz70ml9n18adsk13qb3962-cyrus-sasl-2.1.27/lib/libsasl2.so.3 (0x00007feac4776000) ```

tastytea commented

2020-01-06 15:40:23 +01:00

Your libcurl is built with GnuTLS and mine with OpenSSL, maybe that is it. 🤔 Although theoretically it shouldn't make a difference. Does the file returned by curl-config --ca exist?

Your libcurl is built with GnuTLS and mine with OpenSSL, maybe that is it. 🤔 Although theoretically it shouldn't make a difference. Does the file returned by `curl-config --ca` exist?

FSajadi commented

2020-01-06 19:43:51 +01:00

@tastytea No, the output of curl-config --ca is empty. If I use the latest version of mastodon-cpp I won't have this error. Also the curl -I https://pantherx.social/api/v1/instance worked fine.

@tastytea No, the output of `curl-config --ca` is empty. If I use the latest version of `mastodon-cpp` I won't have this error. Also the `curl -I https://pantherx.social/api/v1/instance` worked fine.

tastytea commented

2020-01-06 20:02:38 +01:00

Seems like libcurl didn't find a certificate bundle when it was compiled. Maybe the command curl uses a different libcurl? I don't know how to fix this, maybe reinstalling libcurl helps.

Seems like libcurl didn't find a certificate bundle when it was compiled. Maybe the command `curl` uses a different libcurl? I don't know how to fix this, maybe reinstalling libcurl helps.

This repo is archived. You cannot comment on issues.

2 Participants

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: tastytea/mastodon-cpp#13