Commit Graph

128 Commits

Author SHA1 Message Date
Azareal cd89e836a1 Some resource management work and performance improvements.
Added the DeallocOverflow and BulkRemove methods to UserCache.
Added the HasUser method to WsHubImpl.
Added a hot path for the common case of a single uncached item in BulkGetMap in DefaultUserStore.
2018-09-13 17:41:01 +10:00
Azareal 1854010840 Fixed a bug where nil sockets weren't getting skipped.
Added some additional sanity checks.
2018-09-12 19:50:57 +10:00
Azareal 646c1f2545 Don't prematurely load topics just to increment their cached view counts. 2018-09-11 22:21:15 +10:00
Azareal 3822fe12d2 Refactored the client-side alert logic.
Exported WsHubImpl's PushMessage method.
Fixed a bug in WSUser's WriteAll and WriteToPage methods where they wouldn't skip over the nil sockets.
2018-09-08 22:47:37 +10:00
Azareal e22a96f53e Oops, make sure those locks are released. 2018-09-08 15:29:51 +10:00
Azareal dfc60ec214 Added support for multiple subscribers mapped to a single user in the WebSockets Component. Untested. 2018-09-08 15:12:48 +10:00
Azareal fd2a35cbcd Topic views should roughly update once every fifteen minutes in the UI now.
Fixed a bug where int64s wouldn't work properly in transpiled templates.
Changed the type of ViewCount in the topic structs to int64.
2018-09-08 11:32:24 +10:00
Azareal 39adc2e606 Per-topic views are now visible in the topic pages for the Nox Theme.
The parent forum of a topic is now visible on the topic pages for the Cosora and Nox Themes.
Added basic styling for polls to the Nox Theme.
Improved the styling for topic title editing in the Nox Theme.
2018-08-30 20:31:21 +10:00
Azareal 41c3a5bb4a Added a Content Security Policy for upgrading insecure images to HTTPS to avoid mixed content warnings. 2018-08-30 15:57:07 +10:00
Azareal f9dedc9691 Fixed a bug with users slipping past the topic list cache. 2018-08-27 19:06:00 +10:00
Azareal 8f7cda9378 Don't double-compress avatars.
Gzip is now disabled for static files when the gains don't justify the costs.
Moved StopServerChan and StoppedServer to package common.
Added some disabled profiling code.
2018-08-22 11:32:07 +10:00
Azareal 7b8943517b Upped the compression level for static files.
Experimenting with compressing everything.
Fixed the live things.

Playing around with topic buttons in Nox.
2018-08-21 18:00:35 +10:00
Azareal 9e8f455bbb Added resource utilisation data to the Debug Page. 2018-08-16 15:27:26 +10:00
Azareal d8682aca94 Move these header sets to the start of the router. 2018-08-15 18:06:48 +10:00
Azareal 3fc2d6a867 Added the init.js file to make certain bits of JS run sooner than others.
The template_*.js files should get cached now.
The phrase API results might get cached now.
Solved an edge case in the attachment parser where a redirect is triggered on a HTTPS site when the attachment is posted as a HTTP URL.
Fixed runHook in the JS files.
2018-08-13 20:34:00 +10:00
Azareal a9a1d667c0 Added the /api/me/ API endpoint.
Moved a script block into an API endpoint.
Split the gigantic switch in ServeHTTP into it's own function.
Scripts should load more efficiently now.
2018-08-12 01:53:42 +10:00
Azareal f2ecb9dbf7 apng is no longer a valid image format as we don't have the technology to optimise it.
Experimenting with systemd.

Began work on an experimental hook table implementation.
Tweaked some hook implementations to make their behaviour less ambiguous.
2018-08-07 14:14:28 +10:00
Azareal b8681524f3 Added a mitigation for polyglot attacks.
.panel_floater is now floated to the right on Nox.
2018-08-06 12:36:24 +10:00
Azareal 01a692ab5b Added the word filter store and moved the word filter routes into the route package.
Added tests for the word filter store.
Added qgen.NewAcc() to reduce the amount of boilerplate needed for creating an accumulator.
Exposed the RecordError method on the accumulator.
Added an Add method to PluginList and removed AddPlugin() in favour of that.

More panel buttons on Nox should be styled now.
Added the panel_update_button_text phrase for future use.

More errors might be caught in the thumbnailer now.
Removed ls from .travis.yml, it was there for debugging Code Climate.
2018-08-04 21:46:36 +10:00
Azareal 8e81f922ea Added some new test cases to catch more things and to see if the statistics on Code Climate change.
Added more TODO ideas for changes.
Removed an unused field from LanguagePack.
2018-07-31 14:02:09 +10:00
Azareal 7691078ce5 Fixed a bug in AddHook where the indices of the hooks would be off by one.
RemoveHook now panics under conditions which should normally never happen, but which when it does, gives off a far more useful message.
Changed the order of the conditional after BypassActive in PluginsDeactivate, so the error message will show up now.
The deactivation handler is optional again for plugins.

Added more tests for the plugin system.
2018-07-29 23:02:48 +10:00
Azareal 50d5be6f32 Moved the plugin manager to the routes package.
Added many, many tests for the plugin system.
Refactored some of the plugin system queries.

Fixed a bug where prepared statements would build up and crash Gosora.
Removed the inline CSS from the plugin rows.
2018-07-29 20:54:12 +10:00
Azareal d74a221467 Move the thumbnailer task closure into thumbnailer.go 2018-07-29 14:34:28 +10:00
Azareal 3aeee419c1 Added support for re-encoding GIFs, hopefully this'll work with animated avatars.
Fixed a bug where jpgs wouldn't get re-encoded.
2018-07-29 00:51:24 +10:00
Azareal 0a628f7201 PNG and JPG avatars are now encoded as JPG images leading to a dramatic drop in the amount of bandwidth used.
Did some work on image thumbnailing, but our dependencies are acting up delaying this from being released.

Fixed the positions of the topic list bits for Nox on mobile.
Removed APNG as an accepted image format, as we don't currently have a good way of optimising these images.
Added a comment regarding the constant time compare for sessions.
Added a warning about putting Gosora in www folders.
Noavatars can now take a width parameters.
Added a bit of missing validation for the avatar uploader.
Refactored the multiple file detector for the avatar uploader.

Added a Run method to accDeleteBuilder.
Added an EachInt method to AccSelectBuilder.
Added a Run method to accInsertBuilder.

Added the users_avatar_queue table, you will need to run the patcher / update script.
You might also want to update the Noavatar field in your config.json file with the new one.
2018-07-28 22:52:23 +10:00
Azareal 60bf821f8a Do constant time compares for sessions for security reasons. 2018-07-20 17:58:59 +10:00
Azareal 75a6ceca84 Added a missing bounds check in peekMatch. 2018-07-18 16:36:16 +10:00
Azareal e5ef86b59c Added user sharding to WsHubImpl to help reduce the amount of lock contention in the upcoming features which will rely heavily on it. 2018-07-18 16:32:48 +10:00
Azareal d0907134ef Database connection drops are handled better now. 2018-07-15 20:29:31 +10:00
Azareal 2a5ab2969c Added Caire and pkg/errors as dependencies.
Made the initialisers and the task runner in main.go easier to debug.

Added form_button_row to a few forms.
Bumped up the attachment image size for Cosora.
Hid the formlabels for the setting editor for Cosora to make it cleaner.
Revamped the account manager with the same CSS as in the Control Panel for Nox.
Started adding the_form to more forms.
Removed the account_emails CSS class from the email editor.
Continued tweaking the Control Panel in Nox to make it look nicer.
Tweaked some of the headers in the Nox Theme.
Added the Create Topic and Moderate options to the topic list on the Nox Theme, although the bulk moderation tools aren't available yet.
Tweaked the padding and sticky shades on the topics on the topic list on the Nox Theme.
Closed topics are now somewhat styled on the topic list on the Nox Theme.
Continued work on the topic pages for Nox.

Renamed Admin Approval to Staff Approval in the English Language Pack.
Added more phrases for the Group Manager and Panel Menu in spots I overlooked.

Began work on the dyntmpl template function.
Be sure to run the patcher / update script to get the new setting.
2018-07-13 21:27:58 +10:00
Azareal 0f2503a20c Made it harder to parse the antispam script.
Used Unicode magic to make it harder to parse the antispam trap question.
Added more shady email substrings.
Tweaked the contributing guidelines, no more await allowed... For now.
Might already have Git, note that.
Added the the_form class to the login form, registration form and login mfa form.
Made the titles on the Control Panel Dashboard and Debug page a little softer on Cosora.

Added the quick topic form to Nox.
Made the topic pages somewhat usable on Nox.
2018-07-05 19:54:01 +10:00
Azareal 3043ce083d Basic proxy IP support.
Switched out a couple of <a>s in the Control Panel with <h1>s, we might want to use <h2> here instead.
Added graphs for Nox.
Tweaked header markup for Nox.
Added support for live topic list to Nox.
Added styling for stickies on Nox.
Fixed topic list text overflow on Nox.
2018-07-03 20:01:49 +10:00
Azareal 2319548e1f Added some heuristics for detecting suspicious emails. 2018-06-30 20:22:39 +10:00
Azareal c640a73afd Fixed a crash bug in the post preparser.
Moved tryStepForward out of PreparseMessage()
Added another test for PreparseMessage()
2018-06-30 14:34:07 +10:00
Azareal bf40d61474 Live Topic List should work properly now with replies too, not just new topics.
Tweaked the visuals for the live topic list on Cosora. More to come with this.
Nox Theme is now present on the theme selector, although it's still under construction.

Improved SEO for social media.
Added the topic_list tmpl phrase prefix.
Added the topic_list.changed_topics phrase.
Added a few more PreparseMessage tests.
Session cookies should now be deleted after you logout, not just blanked.
2018-06-30 13:40:50 +10:00
Azareal 5d810c1e3b We now use SameSite cookies for sessions #security 2018-06-29 14:14:51 +10:00
Azareal 65ceaef4ea Tightened the preparser up a little and wrote more tests to cover more bases.
Green is a little misleading, as we Gosora doesn't call home to check for new versions yet.
2018-06-26 14:54:20 +10:00
Azareal 6eb3429c25 Added i as an alias for em in the HTML parser.
b is now an alias for strong in the HTML parser rather than being it's own thing.
Fixed a bug where tags weren't being closed.
Fixed a bug where the right entities weren't being written all the time.

Added tests for the preparser.
2018-06-26 12:30:29 +10:00
Azareal 1f28ecb804 Rewrote the reverse HTML parser and also fixed the bug with Trumbowyg's span tags sometimes becoming visible. Fixes #9 2018-06-26 00:28:04 +10:00
Azareal 7851724937 / should be mapped properly to the default route for WebSockets events now #11
Updated the README.
2018-06-25 16:24:38 +10:00
Azareal 7be011a30d Almost finished live topic lists, you can find them at /topics/. You can disable them via config.json
The topic list cache can handle more groups now, but don't go too crazy with groups (e.g. thousands of them).

Make the suspicious request logs more descriptive.
Added the phrases API endpoint.
Split the template phrases up by prefix, more work on this coming up.
Removed #dash_saved and part of #dash_username.
Removed some temporary artifacts from trying to implement FA5 in Nox.
Removed some commented CSS.
Fixed template artifact deletion on Windows.
Tweaked HTTPSRedirect to make it more compact.
Fixed NullUserCache not complying with the expectations for BulkGet.
Swapped out a few RunVhook calls for more appropriate RunVhookNoreturn calls.
Removed a few redundant IsAdmin checks when IsMod would suffice.
Commented out a few pushers.
Desktop notification permission requests are no longer served to guests.
Split topics.html into topics.html and topics_topic.html
RunThemeTemplate should now fallback to interpreted templates properly when the transpiled variants aren't avaialb.e
Changed TopicsRow.CreatedAt from a string to a time.Time
Added SkipTmplPtrMap to CTemplateConfig.
Added SetBuildTags to CTemplateSet.
A bit more data is dumped when something goes wrong while transpiling templates now.
topics_topic, topic_posts, and topic_alt_posts are now transpiled for the client, although not all of them are ready to be served to the client yet.
Client rendered templates now support phrases.
Client rendered templates now support loops.
Fixed loadAlerts in global.js
Refactored some of the template initialisation code to make it less repetitive.
Split topic.html into topic.html and topic_posts.html
Split topic_alt.html into topic_alt.html and topic_alt_posts.html
Added comments for PollCache.
Fixed a data race in the MemoryPollCache.
The writer is now closed properly in WsHubImpl.broadcastMessage.
Fixed a potential deadlock in WsHubImpl.broadcastMessage.
Removed some old commented code in websockets.go

Added the DisableLiveTopicList config setting.
2018-06-24 23:49:29 +10:00
Azareal f8f46b3c48 Added support for two-factor authentication.
Added the Account Dashboard and merged a few account views into it.
BREAKING CHANGE: We now use config/config.json instead of config/config.go, be sure to setup one of these files, you can config_default.json as an example of what a config.json should look like. If you don't have an existing installation, you can just rely on the installer to do this for you.

CSS Changes (does not include Nox Theme):
Sidebar should no longer show up in the account manager in some odd situations or themes.
Made a few CSS rules more generic.
Forms have a new look in Cosora now.

Config Changes:
Removed the DefaultRoute config field.
Added the DefaultPath config field.
Added the MaxRequestSizeStr config field to make it easier for users to input custom max request sizes without having to use a calculator or figure out how many bytes there are in a megabyte.
Removed the CacheTopicUser config field.
Added the UserCache config field.
Added the TopicCache config field

Phrases:
Removed ten english phrases.
Added 21 english phrases.
Changed eleven english phrases.
Removed some duplicate indices in the english phrase pack.

Removed some old benchmark code.
Tweaked some things to make the linter happy.
Added comments for all the MemoryUserCache and MemoryTopicCache methods.
Added a comment for the null caches, consult the other caches for further information on the methods.
Added a client-side check to make sure the user doesn't upload too much data in a single post. The server already did this, but it might be a while before feedback arrives from it.
Simplified a lot of the control panel route code with the buildBasePage function.
Renamed /user/edit/critical/ to /user/edit/password/
Renamed /user/edit/critical/submit/ to /user/edit/password/submit/
Made some small improvements to SEO with a couple of meta tags.
Renamed some of the control panel templates so that they use _ instead of -.
Fixed a bug where notices were being moved to the wrong place in some areas in Cosora.
Added the writeJsonError function to help abstract writing json errors.
Moved routePanelUsers to panel.Users
Moved routePanelUsersEdit to panel.UsersEdit
Moved routePanelUsersEditSubmit to panel.UsersEditSubmit
Renamed routes.AccountEditCritical to routes.AccountEditPassword
Renamed routes.AccountEditCriticalSubmit to routes.AccountEditPasswordSubmit
Removed the routes.AccountEditAvatar and routes.AccountEditUsername routes.
Fixed a data race in MemoryTopicCache.Add which could lead to the capacity limit being bypassed.
Tweaked MemoryTopicCache.AddUnsafe under the assumption that it's not going to be safe anyway, but we might as-well try in case this call is properly synchronised.
Fixed a data race in MemoryTopicCache.Remove which could lead to the length counter being decremented twice.
Tweaked the behaviour of MemoryTopicCache.RemoveUnsafe to mirror that of Remove.
Fixed a data race in MemoryUserCache.Add which could lead to the capacity limit being bypassed.
User can no longer change their usernames to blank.

Made a lot of progress on the Nox theme.
Added modified FA5 SVGs as a dependency for Nox.
Be sure to run the patcher or update script and don't forget to create a customised config/config.json file.
2018-06-17 17:28:18 +10:00
Azareal 2d7f302768 Build artifacts are now cleaned up on Windows too.
Tests should be a lot less noisy now.

Fixed a bug where BulkGetMap didn't return any users if you opted out of using a memory cache for the user store.
Used new() in a few more places.
Fixed a test which didn't work properly.
Tweaked the panel_pages_no_pages phrase.
2018-06-06 16:13:55 +10:00
Azareal dd75bf9ad9 All the notices now use AddNotice. 2018-06-06 10:29:12 +10:00
Azareal a5f5f4af7e Added the Page Manager for faster and easier custom page creation.
Added the PageStore.

Renamed account_own_edit.html to account_own_edit_password.html
Renamed custom-page.html to custom_page.html
Renamed the pre_render_custom_page hook to pre_render_tmpl_page.
Added a new pre_render_custom_page hook, not to be confused with the previous one.
Renamed the pre_render_account_own_edit_critical hook to pre_render_account_own_edit_password.
Moved the report forum ID into a constant.
Renamed todaysReportCount to topicsTopicCountByForum and made it more generic.
Renamed panel-menu.html to panel_menu.html
Renamed panel-inner-menu.html to panel_inner_menu.html
Removed an irrelevant editable_parent in a no results row.
Fixed the profile page loading the wrong profile.css
Fixed a bug where the last poster avatar would break on the forum page.
Added the AddNotice method to *Header.
Greatly simplified many of the page struct definitions.
Added the ErrorPage page struct and refactored the error pages to use it.
Added the BasePanelPage page struct and refactored the panel page structs to use it.
Tweaked the DefaultHeader function to set the user on the spot rather than after the fact.
Simplified AccountEditAvatarSubmit into a redirect.
Add the addElement closure in the control panel dashboard to reduce the amount of complexity.
Tweaked LogWarning to better handle nils.

Added the account_username phrase.
Added the account_avatar phrase.
Added the account_email phrase.
Added the panel_pages phrase.
Added the panel_pages_edit phrase.
Added the panel_page_created phrase.
Added the panel_page_updated phrase.
Added the panel_page_deleted phrase.
Added the account_menu_security phrase.
Added the panel_menu_pages phrase.
Added the panel_pages_head phrase.
Added the panel_pages_edit_button_aria phrase.
Added the panel_pages_delete_button_aria phrase.
Added the panel_pages_no_pages phrase.
Added the panel_pages_create_head phrase.
Added the panel_pages_create_name phrase.
Added the panel_pages_create_name_placeholder phrase.
Added the panel_pages_create_title phrase.
Added the panel_pages_create_title_placeholder phrase.
Added the panel_pages_create_body_placeholder phrase.
Added the panel_pages_create_submit_button phrase.
Added the panel_pages_edit_head phrase.
Added the panel_pages_name phrase.
Added the panel_pages_title phrase.
Added the panel_pages_edit_update_button phrase.

Began work on two-factor authentication.
Made more progress with the Nox Theme.
2018-06-06 10:21:22 +10:00
Azareal d897e05256 Users can no longer post or edit posts in locked topics.
Permissions should cascade properly now in the topic template, should have no actual effects on security given the particular nature of this one.
Tiny bit of work on Nox.

Began work on trimming down the page structs to only the necessary parts.
2018-06-01 15:02:29 +10:00
Azareal f85bf51103 Added a couple of simple anti-spam measures.
Made progress with an internal error logging component for the Control Panel.
Made LogWarning less susceptible to races and slightly improved log contention.
Revamped the registration page on Cosora.
Added the SanitiseSingleLine and SanitiseBody functions to better centralise sanitisation and to ensure more consistent sanitisation.
Zero length spaces are no longer permitted in usernames to help prevent impersonation. More to come in this area.
Plugins / internal components can now schedule hourly tasks.
Reduced the chances of newlines breaking the visual layout in areas which expect none.

Added the register_account_anti_spam phrase.
2018-05-31 16:51:31 +10:00
Azareal e311e088c2 Reverted back to URL safe randomly generated strings.
Removed the redundant conditional.
2018-05-28 21:31:19 +10:00
Azareal 4d6a7bfda1 You can now change someone's group again.
The background no longer randomly vanishes on Shadow.
Fixed the small margins on the paginator on Shadow.
Tweaked the padding for the paginator in the Control Panel on Shadow.
The footer is no longer 100% wide on Shadow.
Fixed a misplaced ':' in Shadow.
Added the BlankProfileReply function for tests.
Tests now run once more.
Made it easier to trace test errors which use recordMustExist and recordMustNotExist.

Added tests for profile reply deletion.
Added tests for GeneratePassword in addition to the existing ones for BcryptGeneratePassword.
Added tests for Auth.Authenticate.
Added tests for Auth.CreateSession.

Added a contributor convention for highly unstable builds.
This can be considered a stable build.
2018-05-28 16:27:12 +10:00
Azareal 55c10e0da2 You can now login / change passwords / etc. again.
Fixed the installer.

Shadow theme is still borked.
2018-05-27 20:16:43 +10:00