Logo courtesy of the W3C
The W3C is still grappling with the issues around content transformation. As I wrote back in August, the Mobile Web Best Practices Working Group (BPWG) within the W3C is in the process of formulating a set of Content Transformation Guidelines, defining how web to mobile transcoders should work. The discussions have been extremely heated. Virtually everyone developing mobile web sites just wants transcoders to leave their content alone. There’s considerable distrust in the developer community about the the BPWG, which many feel is stacked in favor of transcoder vendors. It doesn’t help that at least 11 of the 35 BPWG members are employed by companies that build transcoders or carriers that have deployed them. Many also question whether the CTG is even needed as a similar document, Rules for Responsible Reformatting: A Developer Manifesto (The Manifesto) already exists, has received wide support in the developer community and been adopted by four transcoder vendors. The Manifesto was created by a group of mobile web developers on the wmlprogramming mailing list run by WURFL creator Luca Passani.
The latest issue that the BPWG is tackling is how transcoders should handle HTTPS. The problem is that in order to reformat HTTPS content transcoders must decrypt it, breaking end-to-end security.
This is a huge issue and one that is central to the future of mobile services like online banking, shopping, and payments on mobile devices.
On the web, HTTPS is used to provide encrypted end to end connections between the user’s browser and a bank, social network, web-mail service or payment processor’s server. HTTPS is not limited to PC browsers. Direct mobile browsers like Safari, S60Webkit, Android Chrome, Netfront, Opera Mobile or even legacy WAP2 browsers like Openwave and Motorola MIB also support end to end HTTPS connections.
Transcoding proxies like OpenWeb, Infogin and Novarra have to decrypt HTTPS traffic in order to transform it and then re-encrypt it. The single end to end connection becomes two separate secure sessions with a gap in the middle where supposedly encrpted data is in plain text and could theoretically be compromised by someone with malicious intent.
The problem isn’t limited to transcoders like OpenWeb and Novarra either. The increasingly popular server based browsers like Opera Mini, UCWeb, Skyfire, Bolt and Teashark must also decrypt secure data in order to process it.
The core question is whether the two parties in a secure data exchange (you and your bank, for example) are aware of this potential security risk and agree to allow it.
The draft CTG currently say this about HTTPS:
“If the response contains links whose URIs have the scheme https proxies may only rewrite them so that they can transform the content of linked resources, if the following provision is met. If a proxy does rewrite such links, it must advise the user of the security implications of doing so and must provide the option to avoid decryption and transformation of the resources the links refer to.
If a proxy re-writes HTTPS links, replacement links must have the scheme https.
Note: For clarity it is emphasized that it is not possible for a transforming proxy to transform content accessed via an HTTPS link without breaking end to end security.”
That statement does require the end user to be notified, but not the other party to the transaction – the operator of the site that mandated the use of HTTPS in the first place. There have been some critical public comments on the CTG mailing list that seemed to have prompted the BPWG to revisit the topic. In the latest round of public comments at least three approaches have been proposed:
1. Develop additional rules around how and when transcoders should be allowed to intercept HTTPS traffic. For example, only allowing transcoding of sites that have specifically granted permission for transcoders to alter their encrypted data.
2. Prohibit transcoding of HTTPS data streams altogether.
3. Removing all mention of HTTPS from the CTG on the grounds the topic is out of scope and/or too contentious to reach agreement on. This is the approach the Manifesto currently takes, although Luca has made it clear that he believes that transcoders should not touch HTTPS traffic without explicit permission from content providers.
Personally I think that the CTG appears too concerned with the needs of transcoder vendors and not enough with those of end users and content providers. Breaking end to end security is a big deal and should only be permitted with the explicit approval of both parties to the transaction. I feel that the W3c should be soliciting input from the financial industry, especially banks and payment processors. Security of transcoders and server based browsers is not only a technical problem, it’s also social and business problem.
I wonder if there is a financial industry group willing to take on responsibility for auditing the security practices of transcoder and server based browser vendors. Such a group could approve browsers and transcoders and manage a central white-list where content providers could list the URLs of services that they authorize the approved services to transcode.
User permission is also necessary. I’d like to see a warning and prompt presented to the user when they first access an HTTPS site using a transcoder or server based browser. The wording needs to be clear, something like “Do you allow <Name of Proxy> to decrypt the information on <URL> in order to reformat it for you phone? The user should be able to grant blanket or one shot permission.
I trust the major transcoder and server based browser vendors like Opera, InfoGin, Skyfire, Openwave, Bitstream and Novarra. I believe that they are serious about security. However without the buy-in of users and especially financial service providers they are at risk of having their access to these services blocked. It is actually in the best interests of transcoder and browser vendors to be audited by and receive the approval from the secure sites that they are transforming. Doing this on a site by site basis is obviously not scalable but a centralized clearing house could be.
The BPWG is accepting public comments on this topic. If you have an opinion now is the time to be heard. There have been lots of comments from mobile developers and BPWG members but only a couple from anyone whose main interest is online security. It would be great to see more people with experience in online banking or payment processing weigh in on this issue. To be most effective comments should be made on the main topic thread by replying to the message: ACTION-893: Start putting together a set of guidelines that could help address the security issues triggered by links rewriting.
Update: See Tom Godber’s post on this issue. Tom works in the mobile security field and his post explains how https works in a clear and non-technical way and includes an example of how a thief could exploit a transcoded “secure” connection.
This whole issue shows why carriers should be degraded to pure wireless internet pipe providers, and why adoption of IPv6 should be pushed against the stalling based on self-interest of wireless carriers and ISPs.
With every device having it’s on IP address (not a problem with IPv6), there is no more excuse for proxy servers, gateways and other spy-ware, shackle-ware and big-brother-ware in between the user and the content provider.
All I ever wanted from any ISP, wireless or not, is a straight pipe to the internet without so-called “added value” services, which only add value to the ISPs bottom line, not to the user’s internet experience.
The net should be virtual wires, nothing else, unless you specifically and intentionally give up control and make yourself vulnerable by signing up for specific “cloud” services; at which point you get what you deserve.
These proxies exist for the same reason the ISPs fight network neutrality, or want network neutrality with a slew of exceptions attached to the rule: they want to control what goes over the wire, and cripple or stop anything that might advance the universal nature of the internet and cut in their profits derived from legacy services. Best example: the sabotage of ENUM and VoIP.
Nobody should be paying more for a phone line than the cost of a simple domain name registration at this point…
Pingback: Some Lessons from the AT&T/Facebook Switcheroo « EFF
Pingback: Transcoders round 2 | masochismtango
Technologically, breaking HTTPS is needed to reformat protected content. In order to have that checkbox ticked in the comparis matrix, all vendors support the “feature”.
The best way to preserve HTTPS is not to act on transcoder vendors (you would be asking them to amputate their platform), but rather to act on operators who switch the feature on: tell them to stop.
Breaking HTTPS is a major issue that goes beyond transcoding. It brings the whole issue to a new level and exposes operators to non-negligible liability (we are talking about breaking into a secure e2e communication between two parties!).
In a way, this is not different from the fact that virtually all cars are equipped with engines that can make the car go faster than whatever speed limit in whatever country. It is the responsability of the driver to respect speed limits. As far as manufacturers are concerned, they will not implement restrictions on how fast the car can go just because someone asks them, even assuming manufacturers sympathise with that position.
Of course, this would be different if states were to pass laws and regulations which force manufacturers to produce cars that won’t go too fast. Here I see an analogy with W3C mandating that transcoders do not break HTTPS. This could happen if CTG was not at the mercy of vodafoneUK and transcoder vendors.
Luca
Perhaps it’s worth focusing on a single vendor or operator and persuading them to make the switch, then – a single example of this might be persuasive. Are there any you could pursue this with through your Manifesto relationships, Luca?
@Tom
> Is it realistic to expect transcoder vendors
> to remove this feature from their products?
> For operators to un-deploy such software?
> For Opera to stop supporting HTTPS in Opera Mini?
the answers are simple: yes, yes and yes.
Those are abuses and sooner or later abusers will need to back down.
> If not, what’s the best that can be realistically
> achieved?
No need to go for second best. Chances are that developers and content owners will win the first prize. Transcoders are thieves. I can’t see how they are going to get away with what they are doing in the long term.
Luca
My £0.02:
1. If transcoder vendors think this sort of thing can be done safely and securely, let’s hear from them how. As you point out, there are auditing mechanisms for CC payment and perhaps these could be used to provide an industry-facing demonstration that security standards have been applied.
2. There’s piles of difficult UI stuff here: educating mobile customers that security is not the black-and-white state which browsers on the full-fat web show it to be (with e.g. the “padlock” icon showing when you’re connected to an HTTPS server). I’m not sure how you explain this stuff to folks outside the internet/security industries, particularly when on-handset indications of security may not be accurate.
3. This isn’t “going to happen” – it’s happening now. Is it realistic to expect transcoder vendors to remove this feature from their products? For operators to un-deploy such software? For Opera to stop supporting HTTPS in Opera Mini? If not, what’s the best that can be realistically achieved?
4. Caution expressed by large transcoder deployments is worth noting: even operators currently doing HTTPS link-rewriting (e.g. Vodafone) prohibit it for financial services, which would seem to indicate concern over either security or legal issues.
I note that the PCI Security Standards site redirects to an HTTPS version of itself, and might therefore be inaccessible via transcoding; and that it seems to be predominately made up of CC organisations from the Western world. Is there an equivalent for, or is it used in, the Middle East, Asia, Russia, Africa, etc?
Pingback: About Mobility » Blog Archive » The “WAP Gap” All Over Again - W3C, HTTPS and Transcoding
All these assurances about the safety of proxy-mediated split HTTPS links are encouraging (and expected — would any vendor state anything different?), but when everything is said and done, there remain plenty of question marks.
a) Is there any certification authority that systematically validates the security aspects of deployed proxy-based browsing schemes against established norms and publishes the results?
b) Has the compatibility of these proxy-based browsing schemes been settled regarding the conditions stipulated by practically all practically all providers of secure sites, i.e. that access information (e.g. username, passwords, etc) must be kept confidential, and that access must take place under safe conditions? Many of such providers (especially those that deal with financial transactions) indicate that failure to do so may result in blocking or cancellation of the service.
c) Is there any legal basis that defines responsibilities and assigns liabilities regarding the security (authentication, confidentiality, integrity, non-repudiation) of the information flowing through such proxies, especially in view of the fact that their usage occurs in an international setting (i.e. users and proxies are located in different jurisdictions)?
I surmise that the answer is “no” to all answers. I presume that the operators of these proxy-based browsing environments provide licensing and rights-of-use conditions to the end-users that accept no responsibility whatsoever for whatever security problems might occur. Just as providers of terminal browsers.
Finally, let us remember that while the systems at Bitstream (or others) might be inherently better managed and hardened than end-user terminals, they also introduce a single point of failure: breaking into one of their servers, means breaking many end-user communications.
As a matter of fact, the proxy-based browsing approach is eerily reminiscent of the early WAP 1 architecture — with secure communications split between a WTLS and a TLS/SSL link. In those times, all assurances by operators about the excellence of their security management was unconvincing, and resulted in a number of corporations, especially banks, hosting their own WAP gateway in-house.
@Sampo Kaasila,
I like server assisted browsers like Bolt as they consistently deliver a faster and richer experience on mobiles than direct browsers. And I trust Bitstream to protect my identity and my data.
But my trust is based solely on my knowledge that Bitstream is a well established firm with an AFAIK unblemished record when it comes to security. However the fact that the proxy intercepts, decrypts and then re-encypts packets makes it a classic “man in the middle”.
I believe that banks and payment processors will not be comfortable with server assisted browsers unless they have been audited and certified by a recognized 3rd party securiy organization. One such organization is the PCI Security Standards Council (https://www.pcisecuritystandards.org) which certifies online payment processors.
What to you think? Wouldn’t an independent security certification enhance your brand and increase trust of the Bolt browser by both users and financial institutions?
Hi,
I am a member of the BOLT team at Bitstream, and because the BOLT browser is a client/server type of browsing solution I wanted to share some quick thoughts regarding browsing https pages. Lets start with looking at an old-fashioned pure client side browser. Here the user today must trust the browser vendor, since encrypted information is in plain text inside the browser. Additionally, today the user must also trust that the underlying platform, which the browser runs on, has not been compromised by malware. So even with a regular client only browser, the users needs to trust both the browser vendor, as well as the integrity of the platform that the browser runs on.
Now lets contrast this with a client/server type of browsing solution like BOLT. What we have done here is to split the browser into a front-end client component and a back-end server component. Both parts are controlled by the browser vendor; in this case Bitstream. When a user goes to a secure website there is an https connection between the secure site and the backend, and there is another secure connection between our server back end and the client on the handset. Just like with a regular client only browser, information only exists in plaintext inside the browser – the only difference here is that the browser is split into two components, but this does no damage since the communication for https pages is https between the two components. Additionally the backend environment, where the majority of the code runs with a client/server browsing solution, is much more secure than a desktop computer or a handset since: there are physical access restriction at the data center, servers are hardened and patched, and due to ongoing security monitoring and audits.
The user needs to trust the browser vendor no matter if it is a client only browser, or a client/server browser since in both cases the browser has access to encrypted information. It is also interesting to note that in a client/server environment the majority of the code runs in a more secure and hardened environment, than what is typical for client only browsers.
Sampo Kaasila
VP R&D Bistream Inc.
Jo,
I’m sorry that you felt the post was one sided. My intention was to increase awareness of the security implications of transcoding https content and to encourage participation by interested parties in the W3C’s public comment process.
I am certain that there are points of view that I have over looked. Please feel free to add yours to the discussion here in the comments and to encourage other members of the BPWG to do the same.
Hi Dennis
I can’t help thinking that this reads in a rather one-sided way, and I’m wondering how widely you have polled those involved with the process? I’d be more than happy to provide one of several points of view that don’t seem to be represented, if you’d like to contact me.
Jo Rabin
CTO dotMobi
Co-Chair W3C BPWG
Editor W3C Content Transformation Guidelines
Good writeup Dennis. I will make a post on my blog on this important topic.
The Manifesto should be updated to include the HTTPS, since as we can see, the end-to-end secure nature of HTTPS seems that can’t be guarantee for mobile; ridiculous.
I thought the industry already had learned this lesson with the WAP Gap. What is it? A new generation of mobilists already has forgotten about the WAP Gap and need to be re-educated? Or is private interest really this dumb? Don’t break the end-to-end nature of HTTPS! No encrypted content should be altered…
ceo
Great post, Dennis. Very balanced and facts-based, as usual.
Just a quick comment to clarify the Manifesto position with regard to HTTPS.
As you say, the Manifesto does not mention HTTPS. The reason for this is that HTTPS is expected to mean secure end2end HTTP between the server and the client by millions of developers (and users!) around the planet. In a way, the situation is not very dissimilar from a book on “good manners” not mentioning that *stealing is NOT OK* :)
Seeing how the HTTPS discussion has evolved over the past few months, I and others on WMLProgramming have considered explicitly adding HTTPS preservation as a Manifesto rule. Since the Manifesto has gained the support of key transcoder vendors, though, changing the rules along the way is not simple and would require re-negotiating. The prevailing feeling is that, in this time in which Novarra still claims that it’s OK to extort and reformat everything on the web (even already mobile-optimised sites!!!), the Manifesto represents a solid defense against abusive installations, and nothing that may weaken its position should be tried at this stage.
Thank you
Luca Passani