You are viewing bramcohen

Sun, Jan. 29th, 2006, 08:21 pm
Obfuscating BitTorrent

People ask about making an 'obfuscated' version of BitTorrent often enough that it ought to be a FAQ. The reason given is that some ISPs are doing traffic shaping on BitTorrent protocol, and people wish to get around that.

There are several whole categories of reasons for not doing it. First, there's hardly any benefit. Most ISPs don't do such shaping, and attempts at obfuscation won't work for long - the ISP traffic shaping tools are already quite sophisticated, and a wire protocol which transfers a lot of data bidirectionally and consistently looks like line noise with no header is only marginally more difficult to identify then one which uses fixed ports. Obfuscating the protocol doesn't even claim to make it difficult to find out who's downloading a particular file.

Second, if done poorly it has the potential to create an outright incompatibility between obfuscating clients and non-obfuscating clients. This would cause a huge amount of performance problems and general pain, made even more ludicrous by the lack of benefit it brings in the first place. There is in fact a proposal being worked on by some people for an obfuscating protocol which would have exactly this problem. Fortunately it's quite simple to avoid this problem - simply add an extension to the tracker protocol so that a client tells the tracker that it supports obfuscation, and when a tracker gets such a request it returns, in addition to the usual list of peers, a list of peers which support obfuscation. It's very easy for trackers to support such an extension, and it has the benefit of allowing trackers to keep peers from using obfuscation, in case they're interested in making sure the ISP can cache data or just don't want it to be used for some other reason. In general, the tracker should be in control.

That was the main reason I'm writing about this - I rather suspect that some developer has gotten rate limited by his ISP, and is more interested in trying to hack around his ISP's limitations than in the performance of the internet as a whole. Hopefully no such idiocy will take place. Backwards-compatible obfuscation (as opposed to incompatible obfuscation) is still a dubious idea, but at least it isn't a harmful idea.

Back to reasons for not doing obfuscation.

Third, any cacheing which the ISP may do (and yes some ISPs do cache BitTorrent protocol reasonably transparently, much to the benefit of their users) is completely obliterated by obfuscation.

Fourth, when it comes to dealing with ISPs, obfuscation is some combination of hostile, unprofessional, and harmful. Software projects which value quality over featuritis generally steer clear of such things, especially when their potential effectiveness level is the equivalent of spitting in one's face than actual utility.

Oh, and by the way, the amount of CPU necessary to do a diffie-hellman key exchange is enough to be annoying, and if you're making a connection via a trusted intermediary, like, say, a tracker, or already have a reasonably secret piece of shared information like, say, an infohash, there's no need to use a diffie-hellman key exchange to establish a shared secret. Imagining that crypto will stop being done by dilettantes is clearly a pipe dream though.

Mon, Jan. 30th, 2006 05:49 am (UTC)
theducks

People have come up with a transparent caching proxy for BitTorrent? That's quite a frightening concept, but I could see why it would benefit users and ISPs, especially if lots of customers are after the same content.

Tue, Feb. 7th, 2006 12:27 am (UTC)
arafel2

Oh, definitely. Check out the company CacheLogic for an example. They don't just cache BT, either, from what I remember.

Mon, Jan. 30th, 2006 12:39 pm (UTC)
maelorin

also, it breaches your user agreement with practically any isp you careto mention. that is bad. bad i say.

and can lead to nasty consequences for you, the user. especially if they find out.

if not, everyone gets hammered.

see: recent changes to internode adsl plans ... some [***] idiots insisted on playing games to cheat the internode shaping protocols. the 400g/month tards gave us the 90g max cap.

Tue, Feb. 7th, 2006 03:11 am (UTC)
sorbix

i bet if the isp wanted to, they could use their user agreement to sue you for reading an encrypted email or using a secure shopping cart. that doesnt make it right.

seriously, who respects their user agreements?

Sun, Feb. 5th, 2006 06:42 pm (UTC)
jin_n_juice

Hi! Got here from http://torrentfreak.com/?p=117.

First off, a little overdue, but thanks for creating bittorrent in the first place!

Re: "a wire protocol which transfers a lot of data bidirectionally and consistently looks like line noise with no header is only marginally more difficult to identify then one which uses fixed ports."

If properly designed & implemented, wouldn't encrypted BT traffic be virtually indistinguishable from say, a remote user who's got an IPSec tunnel into his office LAN and running something as chatty as Outlook/Exchange?

In that situation, ISPs would have to think a little more carefully about indiscriminately throttling such traffic - or else write it off as collateral damage, which might be dangerous.

Tue, Feb. 7th, 2006 01:33 am (UTC)
xenoxaos

Although an IPSec tunnel is usually directed at a single address, ie. vpn.company.com. If you were seeing traffic going to end users, its easier to see; ip-192.168.isp.net

Sun, Feb. 5th, 2006 08:25 pm (UTC)
hcatlin

As someone who lives behind a shaper (thanks Rogers Cable!), I'd quite like it.

But, perhaps that's my 1.3kb/sec talking.

-hampton.

Mon, Feb. 6th, 2006 11:18 pm (UTC)
okw

Ditto

Shaw isn't as bad though. Was on rogers when living in Mississauga. *shudder*

Mon, Feb. 13th, 2006 12:49 am (UTC)
canoe_drew

Just installed the latest azureus with this feature. Since I'm a rogers customer my DL/UL was 3kps/3kps. Now it's 70kps/90kps. I have to agree with hcatlin: This is fantastic. I don't think the feature is hurting anyone. I can see 7 peers that don't have azureus and they are getting the old 3kps/3kps speeds so the backwards compatibility seems to be working. Of course the best solution would be for rogers to stop shaping the traffic.

Mon, Feb. 6th, 2006 06:46 am (UTC)
cameronbergh: anti filtering measures

I realize that bit torrent was never really intended to be a conduit for piracy, and thus, filtering of the protocol was not forseen. But filtering bittorrent is a trivial joke, and so is the solution to the filtering.

The fact that every singe bittorrent client sets 6881 as the default port has always amazed me. I wonder sometimes if filesharing developers have ever been on a college/corporate network. Im pretty sure that every single corporate firewall in the universe comes with port 6881, 6969, and all other filesharing ports blocked by default. This does not only apply to filesharing, but to any kind of protocol that uses lots of bandwidth (skype, Tor, counterstrike, etc). Is the solution not obvious?

Pretty much all p2p protocols support non default ports, so how hard is it really to generate a random port number upon installation? That would probably eliminate the filtering on 95% of networks. For the remaining 5% that actually do deep packet analysis, A simple Xor (or even cesar shift) encryption would suffice. No need to waste cpu on rc4.

I definatley agree with Bram on the point that this could cause compatability issues. Backward compatability is a must.

Furthermore, Even IF bittorrent was never for illegal purposes, Anti-Filtering teqniques would be very useful to many people. As americans, we realize protocols like fast track and gnutella will be filtered, but something as legit as bit torrent shouldnt be. In places like china and iran, totally legit things are blocked all the time. People there have to tunnel through Tor just to get to google.

We take for granted that we have the (theoretical) right to not be censored, so we dont incorporate anti-censorship functions into our apps. I think these kinds of features would benefit many oppressed people around the world.

This is the way it looks to me. If anyone disagrees, please let me know because i really dont know how anyone else feels on this subject.

PS. Bram is amazing.

-Cameron Bergh

Mon, Feb. 6th, 2006 07:54 am (UTC)
alphasee: Re: anti filtering measures

I agree with almost everything that you've said -- but we should also suppose that bit torrent was never really intended to be a conduit for private transfer/communication either.

The issue inlies with technology and being able to be used -- similar to the concept "Enough monkies on enough typewriters will eventually give you the gilgamesh epic" ... something along those 'guidelines' --

People will use what they have access to. We are creatures that use, and eventually, someone is going to find an even better usage for bittorrent -- or something of the sort.

As for the encryption, and backwards compatibility, there is a header check to find out if the client/communicant has header encryption support (sending a byte string with a length of 0), and if I read the source correctly, it evaluates the client, and then sends data accordingly... good theory on your part though.


If only programmers were perfect ...


Mon, Feb. 6th, 2006 11:34 pm (UTC)
otterley

From the ISP's perspective, the problem is not the BitTorrent protocol per se - they would have similar difficulties with any protocol associated with high data transfers. This is a result of their short-sighted business model, which encourages waste because they charge on a per-month basis for a given pipe instead of charging per bit transferred.

Tue, Feb. 7th, 2006 12:40 am (UTC)
manicdee: Shortsighted?

I don't think you understand the model. Charging per bit transferred is not friendly to end-users - charging for an expected utilisation of bandwidth is much more friendly, since it allows the ISP to plan their bandwidth provision better.

Charging after the fact force the ISP to react to changes in patterns of usage. Charging in advance means the ISP has the extra time to order upgrades where required.

Bandwidth is always going to be wasted - most people don't use the Internet 24 hours a day, not even at an office. Look at the usage graphs for a home user - some very light usage in the morning (checking the web newpapers, morning mail, etc), scattered usage through the day (home machines checking for software updates and keeping NTP clocks up to date), then heavy usage between 4pm and 10pm (between kids coming home from school and going to bed) with a peak at 6pm-8pm.

Ask anyone who has switched from a charge-per-byte plan to a quota-based plan (with shaping rather than excess traffic charges) which they prefer.

Being charged in arrears for volume consumed is a great way to turn yourself into a nervous wreck.

Tue, Feb. 7th, 2006 12:53 am (UTC)
jill23: bandwith issues

Sure ISPs don't like traffic hogs which max out their connectivity. Then why do they penalize their users, thereby forcing them to create countermeasures like obfuscation patches instead of developing BitTorrent in a direction where it tries to use near network nodes first, thereby keeping more of the traffic inside their network rather than using their peering? This would effectively make bittorrent caches out of their local users without them needing to build those themselves.

Mon, Jun. 19th, 2006 02:50 am (UTC)
ex_cloaked902

Some ISP's do have set transfer limits. They suck. I would never use such an ISP, because they overcharge more often then not.

Tue, Feb. 7th, 2006 02:25 am (UTC)
jesuitx

How about the fact that this encryption would prevent people from seeing what you're transferring? It would do that, right?

Freenet is great for these things, but impossible to use for most people. BitTorrent on the other hand is quite easy, and really could benefit from something like this. Particularly if the tracker gets to decided whether or not to allow it, making it totally transparent to the user.

And if "quality over features" really is the motto of the day, then I'd like to know why the trackerless feature, that which every tracker admin disallows, even exists.

Tue, Feb. 7th, 2006 06:59 pm (UTC)
bramcohen

Connection obfuscation doesn't even claim to hide what it is that you're doing. The threat model of someone seeing what you're doing is that they connect to the tracker and get you in the peer list, or you later try to connect to them, none of which is changed by obfuscating the peer protocol. Also, the .torrent file is generally downloaded via unencrypted http, so even a passive snooper on the local network can see what you're doing.

Tue, Feb. 7th, 2006 11:32 am (UTC)
lightyear: Dead right.

There's no point in obfuscating.

Think about it: If the problem goes from being a "known" protocol that they shape, to random noise... and the random noise *is* a problem for them, they're going to shape the random noise.

Sure, that may well put other protocol traffic into the same bin. Sure, that may detrimentally affect lots of other users. But the fact of the matter remains: they thought they had a traffic problem, and once it's all encrypted, they still will: They'll shape down the random noise, and they'll shape down bittorrent.

The difference? they can run a BT cache for standard BT protocol, but the crank down on the noise can't be assisted, analysed, or improved.

If you don't like it, vote with your feet. Not easy for everyone, but it's at least an option. Trying to work around your ISP's filters is going to fail: Any company willing to play hardball with the shapers is going to do the same thing to you again once the usage picks up.

Sure, you might be able to get ahead of the curve - but only for a short while. Then the network administrators will analyse traffic, see the noise, and shape it out of existence, just like your BT traffic.

Push for BT caching - work with them not around them. Or walk away from them, and let them know why. If they don't consider BT a "real" protocol, tell them you're a World of Warcraft player, and 5.5m people worldwide use BitTorrent to download the versions of their games. Take a stand and work with them - but working around them can only ever fail as a long-term strategy.

Wed, Feb. 8th, 2006 06:03 pm (UTC)
newsnightseth: bittorrent

Hi,

I'm a researcher working on Newsnight the UK news and current affairs programme on BBC 2. We are looking for someone who uses a program that encrypts bittorrent (uTorrent, Bit-Comet etc.) to appear in a film about the development of P2P techniques. If you are based in or around London and think you could be suitable send me an email asap with your phone number at sgoolnik@yahoo.com and I will be in touch,

Thanks,

Seth

Wed, May. 3rd, 2006 05:17 pm (UTC)
squirrelitude

I am on a college campus, and torrents are heavily restricted, to the point of uselessness. Some files [like this one] are only distributed as torrents, and if I didn't use an encryption-capable client, it would be impossible to get these files. Encryption is a good thing.

Thu, Jun. 22nd, 2006 09:40 pm (UTC)
shaunburrows

Totally agree with that, was the only way I could use bittorent, so I told every1 else how to do it, was prob responsable for terabytes of upload :P (uni 10Meg up each). Just wished i tried it earier.

Tue, Jul. 25th, 2006 10:08 am (UTC)
kyoguan: let the user choose encrypt or not

Obfuscating must encrypt from the begining, because ipp2p will
filter the URL request to the tracker. Once the client can connect
to the tracker, then the later encrypt will be easy.

Fact bittorrent tracker use the HTTP protocol. Why not add the https support for bittorrent, it will be easy for Backwards-compatible.
for example, now the tracker will use the 6969 as the service port
for HTTP request, may be we can use the 7070 as the service port
for HTTPs request. client can choose encrypt or not, right?

Thu, Nov. 8th, 2007 08:59 am (UTC)
(Anonymous): Perfect Forward Security

Oh, and by the way, the amount of CPU necessary to do a diffie-hellman key exchange is enough to be annoying, and if you're making a connection via a trusted intermediary, like, say, a tracker, or already have a reasonably secret piece of shared information like, say, an infohash, there's no need to use a diffie-hellman key exchange to establish a shared secret. Imagining that crypto will stop being done by dilettantes is clearly a pipe dream though.


The reason is called perfect forward security (http://www.cs.bu.edu/fac/itkis/pap/forward-secure-survey.pdf).

Mon, Apr. 7th, 2008 04:33 pm (UTC)
hasyer: Re: Perfect Forward Security

Thank you. (;

---------------------
muhabbet
korku
msn nickleri
msn nickleri
netlog

Tue, Apr. 14th, 2009 10:31 am (UTC)
bramcohen: Re: Perfect Forward Security

There's this distinction between 'threat model' and 'paranoid delusional fantasy'.