Opened 4 years ago

Closed 3 years ago

#1279 closed New feature (fixed)

Custom ("user") session handler using database (ie MySQL) instead of "files"

Reported by: vipsoft Owned by: vipsoft
Priority: normal Milestone: 1.5 - Piwik 1.5
Component: Core Keywords:
Cc: Sensitive: no

Description (last modified by vipsoft)

Use Zend_Session_SaveHandler_DbTable.

see: http://zendframework.com/manual/1.10/en/zend.session.savehandler.dbtable.html

For Installation, we would store session information in a signed cookie, and delete the cookie when the installation is complete.

Pros:

  • reduce number of inodes, eliminating tmp/sessions
  • move the session handling logic out of ./index.php (to FrontController init?)
  • potentially simplifies the session handling logic
  • no longer second guessing security of user's configuration
  • works with load balanced web servers that don't share session files
  • works when user doesn't have write access to shared session folder (technically, considered to be an improperly configured web server)

Cons:

  • cookies subject to 4K limit (only applies to installer and custom dashboard layout for anonymous)
  • slower (especially with garbage collection)
  • would have to rework LanguagesManager/API.php's use of Zend_Session_Namespace in getLanguageForSession() and setLanguageForSession()

Note: Piwik still requires write access to ./tmp (i.e., templates_c, cache/tracker, latest).

While you're at it, add a systemCheck for the 'session' extension, in case php was compiled with --disable-session.

And look into handling disabled ini_set() -- used by Zend_Session.

Oh, and make sure we handle the case where session.use_cookies = 0 (ref: http://forum.piwik.org/index.php?showtopic=11381)

Attachments (1)

Piwik › Update.png (36.1 KB) - added by matt 3 years ago.

Download all attachments as: .zip

Change History (39)

comment:1 Changed 4 years ago by vipsoft (robocoder)

  • Description modified (diff)
  • Summary changed from Custom ("user") session handler using MySQL instead of "files" to Custom ("user") session handler using database (ie MySQL) instead of "files"

comment:2 Changed 4 years ago by vipsoft (robocoder)

(In [2121]) refs #1279 - refactor session initialization code out of index.php

comment:3 Changed 4 years ago by vipsoft (robocoder)

(In [2122]) refs #1279 - fallback to system default

comment:4 Changed 4 years ago by vipsoft (robocoder)

  • Description modified (diff)

comment:5 Changed 4 years ago by vipsoft (robocoder)

  • Description modified (diff)

comment:6 Changed 4 years ago by vipsoft (robocoder)

  • Description modified (diff)

comment:7 Changed 4 years ago by vipsoft (robocoder)

  • Description modified (diff)

comment:8 Changed 4 years ago by vipsoft (robocoder)

  • Description modified (diff)

comment:9 Changed 4 years ago by vipsoft (robocoder)

  • Description modified (diff)

comment:10 Changed 4 years ago by vipsoft (robocoder)

(In [2280]) refs #1279 - test that session.save_path is also readable, otherwise
session file cleanup may fail, e.g.,

Zend_Session_Exception: Zend_Session::start() - /PATH/libs/Zend/Session.php(Line:480): Error #8 session_start(): ps_files_cleanup_dir: opendir(/var/lib/php5) failed: Permission denied

comment:11 Changed 4 years ago by jreese

I've confirmed on my fresh install that this patch fixes the exception output. Thank you.

comment:12 Changed 4 years ago by vipsoft (robocoder)

  • Description modified (diff)

comment:13 Changed 4 years ago by vipsoft (robocoder)

  • Milestone changed from Features requests - after Piwik 1.0 to 5 - Piwik 1.1
  • Priority changed from normal to major

Reprioritizing. On the demo server, I noticed we have 285000+ session files in piwik/tmp/sessions -- most being empty files. That's a lot of inodes, and could be a problem for some users with shared hosting accounts.

comment:14 Changed 4 years ago by vipsoft (robocoder)

Nice to have:

  • eliminate dependency on libs/Zend/Session and the session extension. (No more workarounds.)

comment:15 Changed 3 years ago by vipsoft (robocoder)

  • Milestone changed from 1.1 - Piwik 1.1 to 1.2 - Piwik 1.2

comment:16 Changed 3 years ago by vipsoft (robocoder)

(In [3514]) refs #1279, refs #1910 - language preference for anonymous users is now stored in a cookie; we can't use this workaround for the dashboard layout because of cookie size limits

comment:17 Changed 3 years ago by vipsoft (robocoder)

(In [3515]) refs #1279, refs #1910 - fix typo

comment:18 Changed 3 years ago by vipsoft (robocoder)

  • Description modified (diff)

comment:19 Changed 3 years ago by vipsoft (robocoder)

  • Description modified (diff)

comment:20 Changed 3 years ago by vipsoft (robocoder)

Other suggestions:

  • not every request requires session data; propose not calling Zend_Session::start() in Piwik_Session; ie lazy init
  • Dashboard: set expiry for the anonymous user's non-persistent dashboard layout, eg $ns->setExpirationSeconds(1800);

comment:21 Changed 3 years ago by matt (mattab)

  • Milestone changed from 1.2 Piwik 1.2 to Feature requests

comment:22 Changed 3 years ago by matt (mattab)

  • Priority changed from major to normal

comment:23 Changed 3 years ago by vipsoft (robocoder)

Using a DB table should avoid the potential race condition experienced with locked session files (see #2296).

comment:24 Changed 3 years ago by vipsoft (robocoder)

(In [4761]) refs #1279 - add session table

comment:25 Changed 3 years ago by vipsoft (robocoder)

(In [4762]) refs #1279 - add update script and bump version

comment:26 Changed 3 years ago by vipsoft (robocoder)

(In [4763]) refs #1279 - fix typo

comment:27 Changed 3 years ago by vipsoft (robocoder)

(In [4764]) refs #1279 - typo

comment:28 Changed 3 years ago by vipsoft (robocoder)

  • Milestone changed from Feature requests to 1.5 - Piwik 1.5

comment:29 Changed 3 years ago by vipsoft (robocoder)

  • Resolution set to fixed
  • Status changed from new to closed

(In [4765]) fixes #1279

comment:30 Changed 3 years ago by vipsoft (robocoder)

  • Resolution fixed deleted
  • Status changed from closed to reopened

CI shows the update fails:

[22-May-2011 19:05:51] PHP Fatal error:  Uncaught exception 'PDOException' with message 'SQLSTATE[42S02]: Base table or view not found: 1146 Table 'piwik_qa.canoo_session' doesn't exist' in /home/www/data/root/jenkins.private/jobs/Piwik/workspace/build/libs/Zend/Db/Statement/Pdo.php:228
Stack trace:
#0 /home/www/data/root/jenkins.private/jobs/Piwik/workspace/build/libs/Zend/Db/Statement/Pdo.php(228): PDOStatement->execute(Array)
#1 /home/www/data/root/jenkins.private/jobs/Piwik/workspace/build/libs/Zend/Db/Statement.php(300): Zend_Db_Statement_Pdo->_execute(Array)
#2 /home/www/data/root/jenkins.private/jobs/Piwik/workspace/build/libs/Zend/Db/Adapter/Abstract.php(479): Zend_Db_Statement->execute(Array)
#3 /home/www/data/root/jenkins.private/jobs/Piwik/workspace/build/libs/Zend/Db/Adapter/Pdo/Abstract.php(238): Zend_Db_Adapter_Abstract->query('DESCRIBE `canoo...', Array)
#4 /home/www/data/root/jenkins.private/jobs/Piwik/workspace/build/core/Db/Adapter/Pdo/Mysql.php(211): Zend_Db_Adapter_Pdo_Abstract->query('DESCRIBE `canoo...', Array)
#5 /home/www/dat in /home/www/data/root/jenkins.private/jobs/Piwik/workspace/build/libs/Zend/Db/Statement/Pdo.php on line 234

comment:31 Changed 3 years ago by matt (mattab)

Interesting!!
Questions:

  • Why move Piwik_Session::start to frontController instead of previous location in index.php?
  • Session purge: what is the default timeout after which sessions will be deleted from the DB to ensure the table doesn't grow out of control?

This is great feature because it will ensure that load balanced Piwik Reporting UI will work fine without requiring to "stick" same users to the same backend server!

comment:32 Changed 3 years ago by matt (mattab)

(In [4766]) Refs #1279 Important to copy paste the table definitions in update files: this ensures that subsequent "ALTER" on these tables will work fine (otherwise, if we change table definition in myisam file in 1.8, and users upgrade from 1.4 to 1.9, the 1.5 upgrade will install the newest version of 1.8 tables which will then fail the 1.8 ALTER table on this table..)

comment:33 Changed 3 years ago by vipsoft (robocoder)

I had to move the Piwik_Session::start() to front controller because the save handler needs the $db handle.

Purge: the frequency of cleanup and lifetime is defined in php.ini (e.g., session.gc_maxlifetime)

re: r4767 Good point. I'll go back and fix those cases in 0.2.10, 0.2.13, and 0.2.27, so we don't set a bad example. ;)

I'm just installing 1.0 now, and going through the auto-update to find out what's causing the problem in comment:30.

comment:35 Changed 3 years ago by matt (mattab)

on the update screen I see an error at the bottom, not important but if easy to hide would be nice :) (see attached file)

Changed 3 years ago by matt (mattab)

comment:37 Changed 3 years ago by vipsoft (robocoder)

Yes, I saw the exception too. This is fixed in r4772. I'm just waiting for confirmation from Jenkins before I close this ticket.

comment:38 Changed 3 years ago by vipsoft (robocoder)

  • Resolution set to fixed
  • Status changed from reopened to closed

Thank you, Jenkins.

Note: See TracTickets for help on using tickets.