News
1. “Breach of privacy”
Yesterday, the province announced that the Freedom of Information site had been “breached”:
Government is working with Halifax Regional Police to investigate a breach of information on the Freedom of Information and Protection of Privacy (FOIPOP) website.
Of the documents that were inappropriately accessed, less than four per cent, about 250, contained highly sensitive personal information. In total, about 7,000 documents were inappropriately accessed.
Examples of sensitive information may include birthdates, social insurance numbers, addresses and government services client information. Applicants’ credit card information was not accessed during the breach.
Government first became aware of the issue on Thursday, April 5, when a problem was identified with the freedom of information website. It was immediately shut down while a solution was sought for the problem. On Friday evening, April 6, government confirmed information was inappropriately accessed. Government filed a complaint with Halifax Regional Police to investigate on Saturday, April 7.
Seven minutes after the provincial announcement, Halifax police issued this statement:
Police investigating breach of confidential electronic Government information
Halifax Regional Police is currently investigating a breach of electronic Nova Scotia Government files.
At 8:30 p.m. on April 7, police received a report from provincial government corporate security advising there had been breach of security on their network. The breach occurred between March 3 and March 5, 2018 and was discovered by a government employee on April 5.
Earlier this morning investigators with the General Investigation Section and Cyber Crime Unit of the Integrated Criminal Division executed a search warrant at an address in Halifax and took a person into custody in relation to the incident. The investigation is ongoing and charges have yet to be laid.
At a press conference, details emerged, reports Jacob Boon:
At a press conference Wednesday, deputy minister of Internal Services Jeff Conrad described how an unauthorized user had — over two days in March — accessed private files held on Nova Scotia’s Freedom of Information web portal.
The hacker, if that word even applies, realized the private PDFs located on the government website could be viewed simply by changing file numbers in the URL. Using a script that sequentially replaced those digits, the individual was able to download the 7,000 documents without anyone noticing.
When I heard that, the first thing I thought was “that could’ve been me.”
See, it took me 25 years of reporting to figure this simple fact out, but all government registries number their documents sequentially. So, suppose I’m on a computer in the courthouse looking at a court filing; if I take the document number and add one, I get the next document filed. Same with corporation filings. Same with property records. And so forth.
I do this all the time, and there’s absolutely nothing illegal about it. These are public documents, and I have every right to access them. The “number trick” makes me a better reporter.
I go to the FOIPOP website nearly every day, and I link to documents from it. It kind of amazes me now that I didn’t realize those documents were numbered sequentially. Take, for example, the document I linked to in January about the Art Gallery of Nova Scotia’s plan for “cultural hub”; here’s the URL:
Because the site is down, the URL doesn’t now work. But you see that “7433” part? Had I been more attentive, I would’ve realized that if I simply changed it to 7434 I would’ve gotten the next document in the sequence. And I would have. You bet I would have. That’s my natural curiosity at work. Now, I’m not technically sophisticated, so I just do the “number trick” manually, and in this case I would waste hour after hour typing the next number into the sequence to get a new document. But if I knew how to write a script to make the process run quicker, I would have, and I would’ve just dumped the documents into a file that I could examine at my leisure.
Does that make me a potential crook? No, of course not. Intent matters. I access public documents with the aim of reporting on them, not to steal identities or defraud people. And if I found material that I knew should not have been public, I would have notified the website owner (this has actually happened twice in my career).
I don’t know what the 19-year-old who was arrested was up to, but my guess is that any privacy breaches were entirely unintended. My guess is that, like me, he saw an easy way to download information that was placed on a public-facing website, and the personal information was a byproduct, not the target. I mean, even I know that if I were up to no good, I’d mask my identity with proxy servers and borrowed IP addresses to avoid detection; obviously this person is technically savvy and could have easily done so.
But even if the arrested person had ill-intent, the cause of the primary cause of the privacy breach doesn’t rest with him, but rather with those charged with building and maintaining the website.
The arrest sure looks like an exercise in misdirection — call it crime so we don’t get blamed for sloppy procedures.
King’s journalism prof Fred Vallance-Jones echoed my sentiments in a sequence of tweets last night:
If I understand what happened, someone cycled through a bunch of URLs and methodically downloaded the content. This kind of “scraping” is commonly done by all kinds of people.
— Fred Vallance-Jones (@Fvjones) April 11, 2018
The main complaint of the government is that this someone obtained information they shouldn’t have had, that the government was supposed to protect per FOIPOP. The charge has to do with the actions of scraping the site, saying that those amount to a crime.
— Fred Vallance-Jones (@Fvjones) April 11, 2018
The criminal code section in question speaks of obtaining, fraudulently or without a right, a computer service, or intercepting a functioning of a computer system, or committing mischief by doing things that don’t appear to have been done here.
— Fred Vallance-Jones (@Fvjones) April 11, 2018
2. Cogswell
The Cogswell Interchange plan was rolled out last night, to positive reviews, reports Lama El Azrak for…. er, what do we call that thing now? Metro? The Star? “Star Metro Halifax” is a mouthful, but that’s how it’s branded.
I don’t have strong opinions about the concept plan one way or the other, except to offer two suggestions:
1. Tear down the casino and build an actual waterfront park. The casino is a dead zone on the waterfront, and removing it would open up the entire Cogswell district. The costs of the dead waterfront, the hellish architecture and soul-sucking parking garage, and the social ills that come with gambling far outweigh whatever money comes in from the thing. Tear it down.
2. Remove the pedway that connects the TradeMart building to the Scotia Square parking garage. Nobody much uses it anyway, and those who do can use a proper crosswalk at street level. Removing the pedway would open up a partial view (mostly, Purdy’s Wharf is still in the way) of the waterfront from upper Cogswell Street, which fits into the philosophy of HRM By Design. But more importantly, it would give some much needed space to the block, which now feels like a dank hallway into a torture chamber.
Besides that, in the wake of recent news that office space assessments are way down, I wonder if the financial side of the plan still makes sense. I suppose the whole thing will be built up with condos; there’s nothing inherently wrong with that, but most likely that will be very high-end housing, not for the commoners.
3. Chinese in Halifax
Canadian Press reporter Brett Bundale takes a look at Halifax’s growing Chinese population.
Views
1. Words
“Do you hear a word or a phrase and think ‘I should remember that, it might come in handy’?” asks Stephen Archibald.
Me too! But I have the attention span of a gnat, so unless I scribble down the bons mots they vanish. My work surface gets littered with little pieces of paper and eventually I’ll do some curation and add the words and phrases to a document on my computer. (The document is called Channel Blurring because that was the first phrase I saved back in 2004 or so).
Now, just like Facebook, I’m giving you access of my private file. For the first time you get a glimpse at what attracts my attention in the word department (“shiny object” was one of the phrases that caught my eye).
You can go to the link to see all of Archibald’s words, but here’s his conclusion:
Around 1970 Time Magazine published three columns of jargon words called a Baffle-Gab Generator. All you had to do was chose a random word from each column to make some contemporary, sounding gibberish. Just for you I’ve produced an up to date generator from my collection of words. Fill your boots.
Government
City
Thursday
Appeals Standing Committee (Thursday, 10am, City Hall) — here’s the agenda.
Cogswell District Engagement Booth (Thursday, 12pm and 6pm, Halifax North Memorial Public Library) — all about Cogswell.
Centre Plan – Discuss Package A (Thursday, 6pm, NSCC Waterfront Campus the FABULOUS RAY IVANY MEMORIAL AND CELEBRATORY CAMPUS) — info here.
Public Information Meeting – Case 21099 (Thursday, 7pm, Cafeteria, Basinview Drive Community School, Bedford) — a thing on Fourth Street in Bedford.
Friday
Community Design Advisory Committee (Friday, 11:30am, City Hall) — here’s the agenda.
Province
No public meetings today or Friday.
On campus
Dalhousie
Thursday
Search List in Online Marketplace: Two User Experience Studies (Thursday, 11:30am, Auditorium, Goldberg Computer Science Building) — Kewen Wu from the University of Saskatchewan will speak.
The Central Limit Theorem in Algebra and Number Theory (Thursday, 2:30pm, Room 319, Chase Building) — M. Ram Murty from Queen’s University will speak.
Department of Urology Research Day (Thursday, 4:30pm, Theatre B, Tupper Link) — keynote speaker Colin P. N. Dinney will speak on “Emerging Therapy for BCG Unresponsive NMIBC.”
New Developments in Mali and the Sahel (Thursday, 6:30pm, Lindsay Room, Halifax Central Library) — a roundtable discussion with Bruno Charbonneau, Shelly Whitman, and David Black.
In the harbour
6am: ZIM Tarragona, container ship, arrives at Pier 41 from Algeciras, Spain
7:15am: Skogafoss, container ship, arrives at Pier 42 from Argentia, Newfoundland
8am: Acadian, oil tanker, sails from Irving Oil for sea
8:45am: Scotian Sea, supply vessel, moves from old Coast Guard base to Pier 9
10am: Skogafoss, container ship, sails from Pier 42 for sea
11am: Morning Clara, car carrier, arrives at Autoport from Southampton, England
Noon: Nolhanava, ro-ro cargo, sails arrives at Pier 36 from Saint-Pierre
4:30pm: ZIM Tarragona, container ship, sails from Pier 41 for New York
Footnotes
Short Morning File today because I have to go to court early.
Notorious for inadequate FOI, and now, not enough POP.
Maybe the provincial government should hire Facebook to create a more secure FOIPOP.
I would add it’s entirely possible to see the sequential ID, write the script to download the files, download them all, and *never know any of them are confidential*. After all, you’re a sane and rational human being, why would private documents be accessible the same way as public documents? But I can think of a dozen reasons to want a local cache of FOIPOP releases, all of them legitimate. Maybe if you started reading them you’d realize, but otherwise you wouldn’t know until the HRP kicked down your door.
Why do I think that this is a first year King’s (or NSCC) journalism student grabbing them as part of broader research into FOIPOP?
On the FOIPOP issue Tim, you say “The arrest sure looks like an exercise in misdirection”
I beg to differ, at the time of the investigation, intent was not known, flight risk was not known; there were too many unknowns to simply knock on the door of the suspect and ask questions. Evidence needed to be gathered and what would you or the opposition party be saying if the once the evidence way analyzed and it was found that a crime had been committed… and the police then had to go looking for the suspect (again) that they had just let go. You would have said that the police were incompetent because they had the suspect in hand at one point and let him go before they understood if a crime had been committed. “A bird in the hand is worth two in the bush” is a statement that holds true.
The Conservatives are are saying that that the Liberals should have immediately informed the public that they believed the FOIPOP data store had been “hacked”. But who really would have benefited from that immediate release of information? The “hack” took place a month ago so anyone with real criminal intent would probably already acted on it. No credit card data was downloaded, so immediate financial risk was not an issue. When the “hack” was first made, the bad guy would have been on needles and pins to see if his “hack” had been found. By just saying the online FOIPOP portal was “down” once the breach was noted, gave the police the necessary time to gather the needed authority to exercise a “legal” search of the suspects residence. Immediately releasing all known facts to the public had the potential to alert the suspect that his “hack” had finally be found and given him a chance to destroy possible evidence (if that person thought he/she had committed a crime in the first place). Releasing the “hack” and ongoing police investigation information immediately could only benefit the suspect.
The Conservatives said the “hack” was “unprecedented”… they must be living under a rock; companies with far greater IT security resources have been hacked before; Apple, Amazon, Yahoo, Microsoft, just to name a few. The reality is that no computer system that is connected to a network (public or otherwise) can ever be assumed to be secure. The IT staff for the province took effective action once the “hack” was noted… the only miss-step in the process was the staff person who first noted the possible “hack”, left a voice message rather than reporting it directly to someone in authority. I see no signs of a cover up.
The Press and the public demanded that the government give them online access to FOIPOP data and the the government did so. That there was a breach was unfortunate; but it was certainly unintended. Everything that is created by mankind has the potential to fail and often does; it is what happens after the failure that is really important. If one wants “easy” access to FOIPOP data, one has to realize that there is a learning curve to providing that access safely and securely. The “hack” was a simple numerical substitution process; a child can see that once it has been pointed out and it took over a year for that vulnerability to be exploited (exploited only once… that remains to be seen)… we should count ourselves lucky. Are more vulnerabilities present in the Province’s online data access portals? Perhaps; but they are unknown to the public at this time.
Was the “hack” really a hack? When one submits a request to access a Provincial data portal, one has to agree to use the portal as intended (I believe). Using a command line interface (script) in not a normal access method, is this a breach of the “user access agreement”? Did the “hacker” access data for which he/she was not authorized to do so and if the person noted they had inadvertently download information that breached “privacy” regulations, is that person obligated, “under the law”, to immediately advise proper authorities? These are questions that will need to be answered in the future. I would suspect that a few changes to the “user access agreement” will occur (for more than just the FOIPOP portal).
But all in all, the Provincial IT team seems to have acted appropriately once the breach was noted and the Minister was correct in not immediately releasing all the facts to the public given an assessment of the immediate risks involved in releasing all the fact immediately. Some political hay will be made by the opposition, but if the shoe were on the other foot, I would be surprised if “their” IT team or Minister would have acted any differently. Conspiracy theories aside, in the end, this might just be a learning experience rather than a crime… I hope so.
But I am sure ALL the “facts” will be laid bare once the online FOIPOP site is back available and every news agency can FOIPOP the poop out of the data store to fish out the archived “facts” on the FOIPOP “hack”.
I don’t think this qualifies as a hack. No secure passcodes were used, no false user names and the over riding fact that they found the guy. Not a hacker I would argue. The elephant in the room to me is why was this so easily done?
I agree.
Just because a vulnerability exists, does not give one the right to take advantage of it. Simple to duplicate or not.
Here is the code required to download all the web pages for the registry of historic places. its written in powershell, which comes with every windows PC sold since 2007.
for($x=1000;$x -le “50000”; $x++){
wget http://www.historicplaces.ca/en/rep-reg/place-lieu.aspx?id=$x -outfile “C:\temp\$x.html”
}
Basically how this works is $x is the variable for the sequential id of the web page we want to grab, the first line says to loop between 1000 and 50000, and increment by 1 every pass. the second line grabs the contents of the site url (inserting the $x) and outputting it as a html file. the Curly braces basically contain the part to loop through.
Would that work on a Mac?
On a Mac you can type curl -O https://foipop.novascotia.ca/foia/views/_AttachmentDownload.jsp?attachmentRSN=%5BX-Y%5D where X and Y are the start and end numbers.
The formatting didn’t work, but the basic idea is curl -O (url), with the number you want to increment over in square brackets like [1-5000].
http://osxdaily.com/2014/02/13/download-with-curl/
See Nick’s Solution for a MAC.
This is trivially easy to do. even if the id’s arnt sequential, if you know a range, you will just get back a 0kb file, which you can ignore.
This discussion threat is starting to read like a page out of “2600: The Hacker Quarterly” magazine. I like it, very instructive… but one should only put this knowledge into practice with caution.
nah – this is basic stuff – not hacking. for this to work, the content has to be publicly published. there is no Misuse, no breaking into systems – you’re basically asking the web server to give you one of everything.
My thought on the breach of privacy as well. If anyone is guilty of a breach of privacy, it is the government and its contractor for appearing to not have any protection of that “confidential” data whatsoever.