The Investigatory Powers Bill, being debated in Parliament this week, proposes the first wide-scale update in 15 years to the surveillance powers of the UK law-enforcement and intelligence agencies.

The Bill has several goals: to consolidate some existing surveillance powers currently either scattered throughout other legislation or not even publicly disclosed, to create a wide range of new surveillance powers, and to change the process of authorisation and oversight surrounding the use of surveillance powers. The Bill is complex and, at 245 pages long, makes scrutiny challenging.

The Bill has had its first and second readings in the House of Commons, and has been examined by relevant committees in the Commons. The Bill will now be debated in the ‘report stage’, where MPs will have the chance to propose amendments following committee scrutiny. After this it will progress to a third reading, and then to the House of Lords for further debate, followed by final agreement by both Houses.

So far, four committee reports have been published examining the draft Bill, from the Intelligence and Security Committee of Parliament, the joint House of Lords/House of Commons committee specifically set up to examine the draft Bill, the House of Commons Science and Technology committee (to which I served as technical advisor) and the Joint Committee on Human Rights.

These committees were faced with a difficult task of meeting an accelerated timetable for the Bill, with the government aiming to have it become law by the end of 2016. The reason for the haste is that the Bill would re-instate and extend the ability of the government to compel companies to collect data about their users, even without there being any suspicion of wrongdoing, known as “data retention”. This power was previously set out in the EU Data Retention Directive, but in 2014 the European Court of Justice found it be unlawful.

Emergency legislation passed to temporarily permit the government to continue their activities will expire in December 2016 (but may be repealed earlier if an appeal to the European Court of Justice succeeds).

The four committees which examined the Bill together made 130 recommendations but since the draft was published, the government only slightly changed the Bill, and only a few minor amendments were accepted by the Public Bills committee.

Many questions remain about whether the powers granted by the Bill are justifiable and subject to adequate oversight, but where insights from computer security research are particularly relevant is on the powers to grant law enforcement the ability to bypass normal security mechanisms, sometimes termed “exceptional access”.

The exceptional access provisions in the Bill take two forms: firstly, ordering organisations to remove encryption and other security protections and secondly, hacking computer systems to obtain information (including compelling organisations to assist with such activities). These powers can be exercised for a wide set of grounds: not just national security but also in the interests of the economic well-being of the UK. Orders served on organisations will also be secret.

However, what makes organisations and security researchers most worried about the provisions are the unintended consequences of such powers being exercised: by obliging companies to make it easier for UK law enforcement and intelligence agencies to conduct surveillance on legitimate targets, it will put the private information of other users at risk of compromise by criminals and foreign governments.

The Investigatory Powers Bill requires a wide range of companies and other organisations to comply with requests of law enforcement and intelligence agencies, including public and private network operators as well as software developers, though the exact scope is not clear (this was one point of criticism raised by the committees examining the Bill).

These requirements even apply to companies outside of the UK, though any attempt to make use of this facility is likely to damage the existing international collaborations between foreign companies and UK law enforcement. No country has before attempted to create such wide-reaching surveillance powers, but there have been past attempts to impose exceptional access provisions on public phone companies, so we can extrapolate from these examples to better understand the wider consequences of the Bill.

What research has shown is that time and time again, surveillance facilities designed for lawful use by intelligence agencies and law enforcement are eventually abused for unlawful purposes.

One of the most notable cases was in 2004, when at least 100 senior members of the Greek government had their phones eavesdropped on by an unknown party who had hacked into the Vodafone network in Greece, exploiting capabilities in the network designed for giving law enforcement access to calls, known as lawful intercept. The lawful intercept capabilities are specifically designed to be hard to detect by both the phone network’s customers and staff, so allowed the attack to remain undiscovered for at least 10 months.

Where the exceptional access provisions in the Investigatory Powers Bill also creates problems is that it prevents companies from implementing state of the art security due to concerns that such techniques might interfere with surveillance. Again, we can learn from phone companies where such restrictions are already in place. Security researchers have developed communication mechanisms that prevent messages from being eavesdropped and remain secure if the network provider is compromised (and maintains some security even if the user’s computer is compromised).

However GCHQ, the UK signals intelligence agency, has prevented the inclusion of these protocols in the standards for the phone network. Without robust security protocols, when the phone network security is compromised there’s nothing to protect users’ phone calls. Companies who want to offer good security will want to avoid basing their operations in the UK in order to preserve their trustworthiness in the eyes of their customers.

Hacking by law enforcement and intelligence agencies (“equipment interference” in the terminology of the Bill) is also problematic because exceptional access is normally achieved by causing equipment to malfunction in ways that bypass security checks. Sometimes this works, but other times the malfunction spreads and causes further harm. A 2012 attempt by the NSA to break into a Syrian Internet company in order to conduct surveillance went wrong, resulting in all of Syria being cut off from the Internet.

The Investigatory Powers Bill permits intelligence agencies to engage in “bulk equipment interference” to simultaneously hack many devices, entire organisations or even whole geographical regions. As more and more critical devices (such as for industrial control, healthcare and transport) are being connected to the Internet, the damage that could result from causing an unknown device to malfunction is increasingly hard to predict.

The Shadow Home Secretary has requested changes to the Investigatory Powers Bill, and the Bill will soon appear before the House of Lords. The legality of the Bill’s data retention provisions is also under consideration by European Court of Justice. The Bill may therefore be amended, dropped or voted down.

However, even without being granted the new powers in the Bill there is much that law enforcement and intelligence agencies can already do. Compared to in-person meetings and using the postal service, electronic communications leave a trail of data which can be invaluable to law enforcement. Even if content is encrypted, metadata (who is talking to whom, how often and how much) is rarely protected and as former NSA General Counsel Stewart Baker said, “metadata absolutely tells you everything about somebody’s life. If you have enough metadata, you don’t really need content”.

Far from “going dark” we are entering a “golden age of surveillance”, provided law enforcement is able to effectively use such information. To do so they need to build greater technical capability and develop effective and efficient international collaborations.

The Investigatory Powers Bill is at best a distraction and at worse actively harmful to this longer term goal.