I think we should at least add those useragents to have them recognized correctly. Do you have a list of useragents?

Just a few so far, but I would like to request some by the vendors...
What shell they provide? Only UserAgent or more Info?
OS? Vendor?
This is for instance is a "Medion TV":
HbbTV/1.1.1 (;Vestel;MB70;1.0;1.0) NetFront/3.5
Not very obvious...

Maybe we could collect info on the currently watched TV channel, too

for the Plugin-List:
If User-Agent contains HbbTV the browser supports it :-)
Shame on the people, who made this standard!

See: http://www.etsi.org/deliver/etsi_ts/102700_102799/102796/01.01.01_60/ts_102796v010101p.pdf
Page 38, 7.3.2.4 HTTP User-Agent header

HbbTV/1.1.1 (<capabilities>; [<modelName>; [<softwareVersion>];
[<hardwareVersion>]; <reserved>)

Attachment: first list
TVUAs.txt

If you need this, please get in touch with professional services to sponsor the feature: http://piwik.org/consulting/

WE probably should have a new report that would be a detailed breakdown of all devices used (phones by type, console by type, TV by type, computers, tablets, etc)

I do not really need this.
I am just interested to get it in - and if it takes a year, I am fine with that. I just like to contribute.

If you can point me into the right direction of what all has to be done, I will see, what I can do.

See the master ticket "improvements to devices detection #3612" for more information!

In 6804733ddcb9fcf72b2df3f796a0d5b228bb0ab2: Adding new DevicesDetection plugin, developed by Clearcode and sponsored by a client of Piwik Professional Services. A beautiful work of engineering, all released under GNU/GPL license!

Fixes matomo-org/matomo#3612

• when enabled, the plugin will add a new submenu: Visitors> Devices
• the new devices report contains NEW reports:
  • Much improved Device type (tracking 'car browser', 'console', 'desktop', 'feature phone', 'smartphone', 'tablet', 'tv')
  • Device brand (tracking more than 100 brands such as Nokia, Nintendo, Lenovo or Apple.
  • Device model (tracking hundreds of phone/console models)
  • Operating System versions (tracking 70 variations of operating systems including Ubuntu vs Kubuntu vs Debian vs Lubuntu vs Xubuntu)
  • Operating System families (Android vs Google TV vs Windows vs Windows mobile vs Mobile gaming consoles)
  • Browser versions
  • Browser families

Refs #3505 There is some basic TV detection included and maybe you can help contribute better detections (see the .yml data files containing the regular expressions in YAML format)

HOW DOES IT WORK

This is quite beautiful system. It is a plugin that disabled by default. when enabled, it will create additional columns in the DB. Also at tracking, it will look at the user agent, and try to match it against one browser we know in the databases. The databases of user agent matching are composed by 3 YML files, parsed by spyc.php into php array.

These 3 YML took dozens of hours of work and testing with dozens of mobile phones and devices for accuracy. We are happy with the result as they should cover > 80% of the devices commonly used. We hope the community will help us build up these YML files and make them better, so we can track accurately 90% or 95% of requests.

The performance overhead is pretty small, but parsing the YML files + running dozens of regex will add some overhead. This is why it is still disabled by default. We will think about how to integrate it in core, in the next few months.

Please let me know if you find any problem with this new awesome code!

@mattab

please add following browser engine for the TV devices: ANTGalio/3.1.1.23.04.09

If you need help for detecting TV - I've tons of log files with user agent strings for televisions. And a lot of regex for TV models are available too.

Best way to identify smart TVs is to check the HbbTV/1.1.1 or HbbTV/1.2.1 sting in the UA

@Diwel we would be very interested in your user agent list of strings for TV devices.

Could you post list of user agent and the expected detection (see our existing test suite).

We'd like for more TV sets and smart TVs to be detected accurately within Piwik!

@mattab

Yes I'll make a separate .yml file for TVs! Could take some time.

But the TVs have a lot more interesting information like the HbbTV version or the screen size in inch or the firmware version. And most of the devices have no real browser but a webkit engine. Additional to that we don't have a normal Opera, it's an embedded Opera and in future there will be webkit instead of presto inside...

TV detection would be a nice addition! If you can keep the number of regex to the smallest, it would be nice (for performance)...

Based on the HbbTV standard for UAs

HbbTV/1.1.1 (<capabilities>; [<modelName>; [<softwareVersion>];[<hardwareVersion>]; <reserved>)

First you have to detect if it's a TV device

preg_match('/HbbTV\/[1-9]{1}\.[0-9]{1}\.[0-9]{1}/i', $_SERVER["HTTP_USER_AGENT"]);

After that you can continue with the yml file for TVs I am working on. This will work for nearly 98% of all TVs

But if the UA contains 'firetv-firefox-plugin' it is not a TV, it's the FireHbbTV Plugin for Firefox and we have to ignore that - this needs to be count as a normal FF.

The reason why i did't use a complete regex are UAs like the Samsung TVs from 2011

 'HbbTV/1.1.1 (;;;;;) Maple_2011'

If we use a complete regex we will lose about 40% of the devices by getting vendor and model name!

May be it will work in the future. For example Samsung changed a lot since 2011

 'HbbTV/1.1.1 (;Samsung;SmartTV2012;;;) WebKit'
 'HbbTV/1.1.1 (;Samsung;SmartTV2013;T-MST12DEUC-1111.1;;) WebKit'

I hope others like Toshiba will follow because they put the capabilities in the wrong position

Wrong: 'Opera/9.80 (Linux armv7l ; U; HbbTV/1.1.1 (; TOSHIBA; 46UL863; 19.3.61.14; 3; ) ; ToshibaTP/1.3.0 (+VIDEO_MP4+AUDIO_MPEG+AUDIO_MP4) ; de) Presto/2.6.33 Version/10.60'
Right: 'HbbTV/1.1.1 (+VIDEO_MP4+AUDIO_MPEG+AUDIO_MP4; TOSHIBA; 46UL863; 19.3.61.14; 3; )'

You can see that there are a lot of difficulties.

Did we want to differ between STBs and TVs? That would be another important decision. If we want that, we have to extend the yml structure.

We'll need to extend the browser vendors by ANTGalio, Maple, Netfront, ... to make it work.

To give you a better overview here some UAs

Sony: "Opera/9.80 (Linux armv7l;  HbbTV/1.1.1 (; Sony; KDL32W655A; PKG3.802EUA; 2013;); ) Presto/2.12.362 Version/12.11"
LG: "Mozilla/5.0 (DirectFB; Linux armv7l) AppleWebKit/534.26+ (KHTML, like Gecko) Version/5.0 Safari/534.26+ HbbTV/1.1.1 ( ;LGE ;NetCast 3.0 ;1.0 ;1.0M ;)"
Grundig: "Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.1 (KHTML, like Gecko) HbbTV/1.1.1 (+PVR;Mstar;OWB;;;)"
Humax: "HbbTV/1.1.1 (+PVR;Humax;iCord Cable;1.01.09;1.0)CE-HTML/1.0 ANTGalio/3.1.1.23.04.09"
Panasonic: "HbbTV/1.1.1 (;Panasonic;VIERA 2012;1.261;0071-3403 2000-0000;)"
Philips: "Opera/9.80 (Linux mips; HbbTV/1.2.1 (; Philips; ; ; ; ) CE-HTML/1.0 NETTV/4.2.0 PHILIPSTV/1.1.1 Firmware/171.56.0 (PhilipsTV, 1.1.1,) en) Presto/2.12.362 Version/12.11"
Technisat: "Opera/9.80 (Linux sh4; U; HbbTV/1.1.1 (;;;;;); CE-HTML; TechniSat Digit ISIO S; de) Presto/2.9.167 Version/11.50"
Inverto: "Mozilla/5.0 (;;;) AppleWebKit/534.6 HbbTV/1.1.1 (+DL+PVR; inverto; IDL-6651N Volksbox Web Edition; 1.0; 1.0;) hdplusinteraktiv/1.0 (NETRANGEMMH;)  CE-HTML/1.0"
TechnoTrend: "Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.4+ (KHTML, like Gecko) Version/5.0 Safari/535.4+ HbbTV/1.1.1 (+DL+PVR;;;;;) CE-HTML/1.0 hdplusinteraktiv/1.0 hdplusSmartTV/1.0 (NETRANGEMMH;) TechnoTrend Goerler/Kathrein"
Lwe: "Opera/9.80 (Linux sh4; U; HbbTV/1.1.1 (+PVR; Loewe; SL150; LOH/3.10;;) CE-HTML/1.0 Config(L:deu,CC:DEU); en) Presto/2.10.250 Version/11.60"

Replying to diwel:

Based on the HbbTV standard for UAs

Are there also other TV sets we could easily detect?

After that you can continue with the yml file for TVs I am working on. This will work for nearly 98% of all TVs

Nice.

Did we want to differ between STBs and TVs? That would be another important decision. If we want that, we have to extend the yml structure.

let's not differ between STB and TVs and consider them as TVs for now.

Thank you for your input, looking forward to your Pull request maybe (ideally you may also add/update the list of user agents that we test for)

Replying to matt:

Thank you for your input, looking forward to your Pull request maybe (ideally you may also add/update the list of user agents that we test for)

Plugin updated and testet by myself. Pull request to the master is done. List for the tests was updated too.

Happy testing :)

In 121e1510fba1eab30f9ea87b0f71d889214b8f7f: refs #3505 (see PR #155) added detection of televisions
added detection of 'LG Browser' and 'Espial TV Browser'
added a bunch of test cases

In a9641b6bb0679eebd27576489bf88c1792c1faa9: Merge pull request #213 from piwik/devicedetectionimprovements

refs #4215 #3505 Improvements to DevicesDetection plugin

There are a lot of tvs detected by DeviceDetector. If there are still some missing, please create new issues for them.