hipstercat/hrss
1
1
Fork 0
Code Issues 3 Pull Requests Releases 3 Wiki Activity

Invalid URL error accurs only with Apache and mod_wsgi #1

New Issue
Closed
opened 2018-11-03 18:34:57 +01:00 by doronbehar · 11 comments
doronbehar commented 2018-11-03 18:34:57 +01:00
Copy Link

I'm experiencing a problem that appears only when I run it with Apache and mod_wsgi. Generally the website runs fine (it looks and feels great BTW, good job!) but in production mode, when I click on the setup button, I don't get forwarded to the elements choice page - nothing happens and I can see this debug message in my apache logs:

[Fri Nov 02 18:24:28.904247 2018] [wsgi:error] [pid 16096] [remote XXXXXXXXXXXXXXXXXXXX] [2018-11-02 18:24:28,903] [DEBUG] [utils] https:/docs.djangoproject.com/en/2.1/howto/deployment/wsgi/modwsgi is not a valid url

From some reason the double / after https becomes a single / and the URL is marked as invalid. There is no error message or something alike printed in the website telling the user there is a problem with the URL so that's another problem.

Here is the relevant part of my httpd.conf:

<VirtualHost _default_:80>
    ServerName hrss.doronbehar.com
    Redirect permanent / https://hrss.doronbehar.com/
</VirtualHost>


<VirtualHost _default_:443>
    ServerName hrss.doronbehar.com
    ErrorLog /var/log/httpd/hrss.info-error_log
    CustomLog /var/log/httpd/hrss.info-access_log common
    SSLEngine on
    BrowserMatch "MSIE [2-5]" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0
    <FilesMatch "\.(cgi|shtml|phtml|php)$">
        SSLOptions +StdEnvVars
    </FilesMatch>
    SSLCertificateFile /etc/letsencrypt/live/doronbehar.com/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/doronbehar.com/privkey.pem
    Include /etc/letsencrypt/options-ssl-apache.conf
    WSGIScriptAlias / /usr/share/webapps/hrss/hrss/wsgi.py
    WSGIDaemonProcess hrss.doronbehar.com python-path=/usr/share/webapps/hrss/venv/lib/python3.7/site-packages:/usr/share/webapps/hrss
    WSGIProcessGroup hrss.doronbehar.com
    <Directory /usr/share/webapps/hrss/hrss>
        <Files wsgi.py>
            AuthType Basic
            AuthName "Protected site"
            AuthUserFile /usr/share/webapps/hrss/.htpasswd
            Order deny,allow
            Allow from all
            Require valid-user
        </Files>
    </Directory>
    Alias /static/ /usr/share/webapps/hrss/web/static/
    <Directory /usr/share/webapps/hrss/web/static/>
        Require all granted
    </Directory>
</VirtualHost>
I'm experiencing a problem that appears only when I run it with Apache and `mod_wsgi`. Generally the website runs fine (it looks and feels great BTW, good job!) but in production mode, when I click on the setup button, I don't get forwarded to the elements choice page - nothing happens and I can see this debug message in my apache logs: [Fri Nov 02 18:24:28.904247 2018] [wsgi:error] [pid 16096] [remote XXXXXXXXXXXXXXXXXXXX] [2018-11-02 18:24:28,903] [DEBUG] [utils] https:/docs.djangoproject.com/en/2.1/howto/deployment/wsgi/modwsgi is not a valid url From some reason the double `/` after `https` becomes a single `/` and the URL is marked as invalid. There is no error message or something alike printed in the website telling the user there is a problem with the URL so that's another problem. Here is the relevant part of my `httpd.conf`: ``` <VirtualHost _default_:80> ServerName hrss.doronbehar.com Redirect permanent / https://hrss.doronbehar.com/ </VirtualHost> <VirtualHost _default_:443> ServerName hrss.doronbehar.com ErrorLog /var/log/httpd/hrss.info-error_log CustomLog /var/log/httpd/hrss.info-access_log common SSLEngine on BrowserMatch "MSIE [2-5]" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0 <FilesMatch "\.(cgi|shtml|phtml|php)$"> SSLOptions +StdEnvVars </FilesMatch> SSLCertificateFile /etc/letsencrypt/live/doronbehar.com/fullchain.pem SSLCertificateKeyFile /etc/letsencrypt/live/doronbehar.com/privkey.pem Include /etc/letsencrypt/options-ssl-apache.conf WSGIScriptAlias / /usr/share/webapps/hrss/hrss/wsgi.py WSGIDaemonProcess hrss.doronbehar.com python-path=/usr/share/webapps/hrss/venv/lib/python3.7/site-packages:/usr/share/webapps/hrss WSGIProcessGroup hrss.doronbehar.com <Directory /usr/share/webapps/hrss/hrss> <Files wsgi.py> AuthType Basic AuthName "Protected site" AuthUserFile /usr/share/webapps/hrss/.htpasswd Order deny,allow Allow from all Require valid-user </Files> </Directory> Alias /static/ /usr/share/webapps/hrss/web/static/ <Directory /usr/share/webapps/hrss/web/static/> Require all granted </Directory> </VirtualHost> ```
hipstercat commented 2018-11-05 11:39:12 +01:00
Owner
Copy Link

I have successfully managed to reproduce this issue using Apache2+mod_wsgi and can confirm the issue. However, it is not related to HRSS but to Apache2 which removes duplicate slashes (like in http://) because of the following RFC 1630 :

PATH
The rest of the URI follows the colon in a format depending on the
scheme. The path is interpreted in a manner dependent on the
protocol being used. However, when it contains slashes, these
must imply a hierarchical structure.

Since the slashes are implying a hierarchical structure is /foo//bar is
equivalent in the hierarchy to /foo/bar.

Apache is implementing this RFC statement very carefully, unlike nginx which is more permissive.

The issue would be resolved by using POST requests instead of GET requests which would mean the links as in /setup/<url> would not be shareable anymore (is it really a problem?).

I have successfully managed to reproduce this issue using Apache2+mod_wsgi and can confirm the issue. However, it is not related to HRSS but to Apache2 which removes duplicate slashes (like in `http://`) because of the following RFC 1630 : ```text PATH The rest of the URI follows the colon in a format depending on the scheme. The path is interpreted in a manner dependent on the protocol being used. However, when it contains slashes, these must imply a hierarchical structure. ``` Since the slashes are implying a hierarchical structure is /foo//bar is equivalent in the hierarchy to /foo/bar. Apache is implementing this RFC statement very carefully, unlike nginx which is more permissive. The issue would be resolved by using POST requests instead of GET requests which would mean the links as in `/setup/<url>` would not be shareable anymore (is it really a problem?).
doronbehar commented 2018-11-05 17:37:02 +01:00
Author
Copy Link

Wow, well done on the investigation! As for your question, using POST instead of GET won't matter too much for me. Not having the ability to use the /setup/<url> links is a price I'm willing to pay. I wonder though if there's a workaround for that..

Wow, well done on the investigation! As for your question, using `POST` instead of `GET` won't matter too much for me. Not having the ability to use the `/setup/<url>` links is a price I'm willing to pay. I wonder though if there's a workaround for that..
doronbehar commented 2018-12-19 12:47:12 +01:00
Author
Copy Link

Hello again, @hipstercat.

Today I've encountered a web app written in Go called bugzillatoatom which does a similar job to hrss. I noticed that it accepts a URL in a form and redirects you to an RSS page. I think it uses POST but it converts the special characters like / and : in the input URL to their corresponding URL percent codes.

Do you think this method could be used in Hrss as well? This should work as a workaround for this problem as well right?

Thanks.

Hello again, @hipstercat. Today I've encountered a web app written in Go called [bugzillatoatom](https://github.com/kaueraal/bugzillatoatom) which does a similar job to hrss. I noticed that it accepts a URL in a form and redirects you to an RSS page. I think it uses `POST` but it converts the special characters like `/` and `:` in the input URL to their corresponding URL percent codes. Do you think this method could be used in Hrss as well? This should work as a workaround for this problem as well right? Thanks.
hipstercat commented 2018-12-19 13:24:51 +01:00
Owner
Copy Link

Autoclosed the issue by mistake. I implemented this workaround in commit 7872bd31af

Does it work on your end now?

Autoclosed the issue by mistake. I implemented this workaround in commit https://hipstercat.fr/gogs/hipstercat/hrss/commit/7872bd31af921978b2c22b56c79dabd1f649476c Does it work on your end now?
doronbehar commented 2018-12-19 15:33:23 +01:00
Author
Copy Link

The redirect to the setup page works well with the encoded URL but I don't see any HTML inside the element selection.

The redirect to the setup page works well with the encoded URL but I don't see any HTML inside the element selection.
NUCselection:2018.12.19-16:31:05.png
101 KiB
NUCselection:2018.12.19-16:31:05.png
hipstercat commented 2018-12-19 15:39:03 +01:00
Owner
Copy Link

Should be fixed by 64f6e33b34

It is okay for you?

Should be fixed by https://hipstercat.fr/gogs/hipstercat/hrss/commit/64f6e33b348a19abb9eda527a11f77c82bce05f1 It is okay for you?
doronbehar commented 2018-12-19 23:28:25 +01:00
Author
Copy Link

Still the same.

Still the same.
hipstercat commented 2018-12-20 11:55:57 +01:00
Owner
Copy Link

The iframe URL is still not urlencoded like this?

The iframe URL is still not urlencoded like this?
hrss.png
133 KiB
hrss.png
doronbehar commented 2018-12-20 13:30:15 +01:00
Author
Copy Link

It's encoded OK. The problem wasn't that, It was a Cross origin resource sharing denial. Here's an error message from the element inspector:

Load denied by X-Frame-Options: https://hrss.doronbehar.com/iframe/https%253A%252F%252Fhipstercat.fr%252Fgogs%252Fhipstercat%252Fhrss%252Fissues%252F1 does not permit framing.

I've partly solved by adding to my httpd.conf:

Header set X-Frame-Options allow

Which removed the error message but from some reason the webpages inside the frame don't have any CSS applied to them:

It's encoded OK. The problem wasn't that, It was a Cross origin resource sharing denial. Here's an error message from the element inspector: ``` Load denied by X-Frame-Options: https://hrss.doronbehar.com/iframe/https%253A%252F%252Fhipstercat.fr%252Fgogs%252Fhipstercat%252Fhrss%252Fissues%252F1 does not permit framing. ``` I've partly solved by adding to my `httpd.conf`: ``` Header set X-Frame-Options allow ``` Which removed the error message but from some reason the webpages inside the frame don't have any CSS applied to them:
selection:2018.12.20-14:28:52.png
122 KiB
selection:2018.12.20-14:28:52.png
doronbehar commented 2018-12-20 13:59:14 +01:00
Author
Copy Link

And that was solved by adding to httpd.conf:

Header always set Strict-Transport-Security "max-age=63072000; includeSubdomains;"

I see this issue as solved, unless you have anything to add?

And that was solved by adding to `httpd.conf`: ``` Header always set Strict-Transport-Security "max-age=63072000; includeSubdomains;" ``` I see this issue as solved, unless you have anything to add?
hipstercat commented 2018-12-20 14:32:11 +01:00
Owner
Copy Link

Looks like you had Apache options which conflicted with HRSS. X-Frame-Options is a HTTP header which restricts the use of iframes. The default is to permit iframe on most websites, unless specified by this directive. This issue happens because of extended security you have set on your Apache installation, which has nothing to do with HRSS unfortunately.

As a result, I can indeed close this ticket. Thank you for the reporting the initial issue.

Looks like you had Apache options which conflicted with HRSS. X-Frame-Options is a HTTP header which restricts the use of iframes. The default is to permit iframe on most websites, unless specified by this directive. This issue happens because of extended security you have set on your Apache installation, which has nothing to do with HRSS unfortunately. As a result, I can indeed close this ticket. Thank you for the reporting the initial issue.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: hipstercat/hrss#1
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?