We put the IT in city®

CitySmart Blog

Tuesday, June 6, 2017
Ryan Warrick, Network Infrastructure Consultant

Ryan Warrick For as long as you’ve used email over the course of your life, you’ve also had to deal with spam email. Like weeds in a garden, spam seems like an inevitable part of using email. Why is it so hard to stop spam? That’s a great question, and it takes us into some important aspects of email that can give anyone a headache. While spam is here to stay (for now), there are some ways to lessen the “weeds” in your email garden.

3 Reasons Why Spam Is Hard to Stop

Spam is hard to stop mostly because of flaws with underlying email technologies combined with the persistence of professional fraudsters.

1. Email addresses are easy to spoof.

Have you ever received a spam email from one of your friends or colleagues? Or an email that seems like it comes from a familiar company? Spammers can set up servers and use software that help them create emails appearing as if they come from a legitimate email address.

While some progress has been made with ways to combat email spoofing, many email service providers and organizations hosting their own email servers don’t use these methods. Plus, spammers often stay ahead of the game by using better and better email spoofing technology and techniques.

2. Email filtering is never an exact science—and always a problem.

In a perfect world, every single legitimate email would land in your inbox and every single spam email would land in your spam folder. But that’s nearly impossible due to the imperfections involved in filtering emails. When emails get filtered, they are filtered automatically based on rules both automatically and manually set up.

The good news? Most spam never even makes it to your spam folder, so email service providers keep getting better with filtering. But for those less obvious spam emails that make it to your spam folder, it’s not uncommon to find legitimate ones. That’s why you need to occasionally check your spam folder to make sure you’re not missing legitimate messages—and also why an occasional spam email may make it into your inbox.

3. Spammers keep adapting as technologists improve antispam techniques.

Recently, the US Justice Department coordinated with Spanish law enforcement to arrest Peter Yuryevich Levashov—a Russian spammer whose operation infected approximately 100,000 computers around the world. According to Wired, “Levashov had long run the Kelihos botnet, a global network of infected computers that collectively flooded email inboxes worldwide with spam, stole banking credentials from infected users, and spread malware across the internet.”

We include this example to show that professional fraudsters are often behind most spam. As a form of organized crime, these fraudsters run sophisticated operations and know what they’re doing. That’s why it’s hard to eliminate spam entirely. These criminal professionals constantly evolve their spam techniques, learn what works and what doesn’t, and adapt.

3 Ways to Weed Out Spam

Despite these issues, it’s still possible to help weed out spam and lessen its impact. Here are three things you can do.

1. Use an enterprise-grade antispam solution.

Depending on the email solution you use, your antispam may not be up to the task of combatting spam—especially if you’re relying on a consumer-grade or manual solution. With an enterprise-grade email solution monitored and maintained by IT professionals, you will have much stronger antispam capabilities that keep you more secure.

2. Train users not to open or click on suspicious attachments.

Even the best antispam solution can’t stop a city employee from clicking on a suspicious email attachment. In 2016, the Verizon Data Breach Investigations Report noted that “30 percent of phishing emails get opened.” If 30 percent of your employees are likely to open a spam email, then you need to offer training and communications about the dangers and liability of opening suspicious email attachments.

And even if users simply open a spam email to look at it without clicking on anything, it tells the spammers that you opened it. Spammers see an “open” as a sign of interest, and they will send more spam emails your way.

3. Tell users not to share their work email address on the internet.

Many spammers get email addresses off the internet by “scraping” websites. Also, you may get a lot of legitimate but still annoying spam when you share your email address on websites for various reasons (such as shopping online, interacting with businesses, subscribing to online publications, etc.). Many legitimate organizations sell your email address to third parties that will barrage you with marketing emails.

Like weeds, spam will never fully go away but you can take steps to lessen its impact. If you’re struggling with spam and need help with some “weeding,” reach out to us today.

Tuesday, May 30, 2017
John Miller, Senior Consultant
John Miller

In the news, we’ve seen plenty of times when government employees get into a lot of trouble by using software that’s not approved by government entities. From private email servers to encrypted messaging apps, big problems occur when government employees download software outside of IT policy.

As a recent article by Governing points out, the risks of “unsanctioned software” or “shadow IT” ripples all the way down to local government. According to the article:

Security is the biggest problem with shadow IT. Whether the software is American or foreign, it often doesn’t meet the strict security standards set by government cybersecurity protocols. Popular file-sharing apps, for example, allow users to easily upload, store and download files, but they may contain viruses or malware that can spread and infect a state government network.

Plus, it’s easier to install software nowadays. With so much cloud software dominating our lives, city employees usually don’t need to purchase physical software, stick a CD into their computer, and install it. Cloud software is ready to go in seconds and…boom! Employees start using it immediately.

While downloading such software may be fine at the employee’s home, remember that you’re an important government entity—a municipality that needs to protect critical citizen information and comply with important laws.

The Governing article gives a great overview of the problem but doesn’t go into many security specifics about why you need to guard against city government employees who download unauthorized software. Here are 7 questions to ask yourself about this software.

1. Who is patching and updating the software?

Software needs regular patching to fix bugs and security holes along with updates to improve performance. With authorized software, your IT staff or vendor oversees this updating and patching. If an employee downloaded the software, then critical security holes could stay open to attackers for months.

And even if employees think the software automatically updates, it’s not unusual for something to go wrong. Who is checking for this? Who is hoping things will go wrong?

2. How do you know you haven’t downloaded a virus or malware?

Employees mistakenly downloading viruses and malware—including from downloading malicious software—remains one of the leading ways that cities suffer disruption and permanent data loss. This is especially a risk when employees download lesser known software that looks appealing on the surface but is riddled with malware or viruses—giving hackers a back door to your city.

You might say, “But my employees only use well-known software.” Even if that’s the case, downloading software on their own still introduces risk. We told a story a few years ago about a tech-savvy colleague of ours who, while not a IT professional, has been involved in the information technology field for over 10 years. He downloaded what he thought was a well-known internet browser that looked like it was from a legitimate website and ended up downloading a virus. So even for “common” software, don’t take the risk.

3. What happens if your employee needs helpdesk support?

Let’s say your employee runs into a problem with an unauthorized cloud spreadsheet application. The file got corrupted somehow and then they lost access to it. Well...it’s not authorized software. Your IT staff or vendor may try to help, but success is not guaranteed.

Why? When your IT staff or vendor supports authorized software, they have installed it, updated it, patched it, maintained it, monitored it, and established a relationship with the vendor. That’s why they can easily help with authorized software problems. None of that knowledge and support framework exists with unauthorized software. When it runs into problems, you’re pretty much stuck.

4. Are you sure that your employee isn’t breaking the law?

This problem crops up with software that stores documents and communications outside of official city government channels. When you receive an open records request, then what do you do if employees are using personal cloud software like Google Docs, Yahoo email, or a file-sharing service like Dropbox. Bring out the lawyers. You’ll need them.

More importantly, these documents and communications may not follow city government security standards. A state like Arkansas is now legally permitted to take away a city’s charter for such security gaps—and other federal and state laws look like they will eventually follow suit.

5. What happens if you lose data?

While an employee may take the initiative to back up data stored on unauthorized software, don’t hold your breath. It’s probably not happening, not happening frequently enough, or not being tested to make sure they can restore data if it’s lost. By contrast, authorized software is usually backed up professionally and overseen by IT staff or a vendor.

6. Do unauthorized people have access to data?

Government data within applications such as financial software, document management systems, and email is usually locked down and only accessible by authorized users—with user access managed by your IT staff or vendor following strict policy. With unauthorized software, who has access to sensitive data? What if your employee accidentally publicly shares a Dropbox link to documents containing sensitive information? Are you seriously relying on the individual judgment of one employee using unauthorized software rather than locking down authorized software that follows a city-wide policy?

7. What happens when software conflicts with the employee’s machine or device?

On a more tactical level, people often do surprising things when they download software. If they have an old desktop or laptop, they may download new software that the machine or operating system just can’t handle. Then, their computer breaks and guess who they call in a panic? Your IT staff or vendor.

We know. This is a tough problem to solve. It’s hard to police the use of authorized software and root out all unauthorized software. While the problem may never fully go away, you can:

  • Create a clear policy about unauthorized software and the consequences for using it.
  • Provide a reminder about security risks such as data breaches, permanent data loss, and breaking the law.
  • Provide a list of approved, authorized software and a contact number for questions if employees want to confirm the use of a particular kind of software.

Think you have a problem with unauthorized software at your city? Reach out to us today. We can help.

Tuesday, May 23, 2017
Nathan Eisner, COO
Jessica Zubizarreta

If ransomware hasn’t gotten your attention yet, then the WannaCry ransomware cryptoworm that ravaged the world for a week in mid-May should make you sit up. The attacks were so devastating to many organizations—from major hospitals to important financial institutions—that ransomware is now mainstream news and the talk of federal and state legislators.

WannaCry 101: Getting You Up to Speed

You may have seen a lot of headlines and articles about WannaCry, but here are the basics to get you caught up.

  • WannaCry is the name of a specific “ransomware cryptoworm.” Ransomware is a type of virus that encrypts your files and documents. The criminal then asks for a ransom within a specific time period (such as 72 hours). If you pay, then they (may) decrypt your files. If you don’t, you permanently lose access to those files. A cryptoworm is a self-replicating virus that encrypts files—meaning that once the virus in inside your IT systems, it can infect other machines without any city employee doing anything.
  • WannaCry originated from a leak of National Security Agency (NSA) data that indicated a security vulnerability in Microsoft Windows operating systems. Hackers stole this information from the NSA and used it to create the ransomware cryptoworm.
  • WannaCry had its biggest impact from May 12-19, 2017 when it affected about 230,000 computers across 150 countries.

Why Your City May Be in Serious Danger from a Future Ransomware Attack

While the media outlined the sophistication and wide reach of this attack, it mostly hit organizations that did not follow three important technology best practices.

This is important for cities to realize: It’s likely that your city has a good chance of experiencing a devastating ransomware attack that leads to permanent data loss if you don’t follow the three best practices below.

1. Failing to regularly patch your software.

Microsoft released a Windows security patch in March 2017 that prevented WannaCry from affecting an organization. According to CNN, “The ransomware is spread by taking advantage of a Windows vulnerability that Microsoft (MSFT, Tech30) released a security patch for in March. But computers and networks that hadn't updated their systems were still at risk.”

Yet, so many organizations—including cities—do not patch their software on a regular basis. Excuses are plentiful. City staff have too much on their plates. Reactive IT vendors do not get paid to do proactive IT maintenance. Nothing appears broken, so why fix it? It’s not a priority. Et cetera.

But when you don’t regularly patch, you miss out on security updates. Software vendors plug holes that hackers can exploit. When you don’t apply patches, it’s like leaving a back door open in your house. Organizations that did not apply the March 2017 Microsoft patch left this back door wide open.

2. Failing to update your operating system.

WannaCry devastated organizations using outdated, unsupported operating systems such as Windows XP, Windows Server 2003, and Windows 7. A newer operating system like Windows 10 wasn’t affected by WannaCry at all.

If your city is running an outdated Windows operating system, consider that:

The older an operating system becomes, the more security issues it will have and there is less of a chance that Microsoft will provide security patching. Many organizations—including cities—stick with older operating systems because of poor practice, older software that’s only compatible with older operating systems, and an unwillingness to budget for the upgrade of operating systems.

Think of your operating system like a car. If Microsoft has stopped supporting it, it’s like driving a car that no professional will officially or possibly be able to repair anymore. You’re essentially just stitching it together with band-aids and waiting for it to break down, at any time.

3. Failing to modernize your technology and get rid of legacy systems.

This issue has become so prevalent across federal, state, and local government that proposed legislation such as the Modernizing Government Technology (MGT) Act specifically addresses IT modernization. In 2017, there is no longer a “nice-to-have” argument about modernizing technology. Instead, modernized technology and cybersecurity are increasingly seen as one and the same thing. The recent WannaCry attacks are now referenced by legislators pushing IT modernization bills—and they see it as both a national security and citizen privacy/protection issue.

For cities, it will become more and more negligent to cling onto old legacy hardware and software that uses obsolete, unsupported, and unsecure technology. While budget is always a concern, the costs of a cyberattack—financially, legally, and politically—can be far worse. States such as Arkansas have even passed laws threatening to revoke a city’s charter if they don’t comply with the law through using appropriate, secure technology.

While the WannaCry attacks might look scary, they really only affected organizations that failed to implement basic IT best practices such as patching, using fully supported Windows operating systems, and keeping their technology modernized.

If your city isn’t following the three best practices above, you are at risk for a ransomware attack. Reach out to us today with any concerns.

Monday, May 15, 2017
Dave Mims, CEO
Dave Mims

Even if your city is not located in Arkansas, it’s still worth noting that the state’s Senate Bill 138 was signed into law by Governor Asa Hutchinson on March 29, 2017. For a quick recap on the law, read our March 8 blog post where we summarized and tracked the law while it was going through the state’s House and Senate.

The passing of this bill is important to cities for a few reasons.

1. Arkansas cities can now lose their charter from non-compliance with IT-related accounting practices.

Arkansas already has a Municipal Accounting Law (§ 14-59) that requires compliance with accounting best practices and includes penalties for non-compliance. But now, Senate Bill 138 adds some teeth to the law by clarifying that not following specific IT-related accounting best practices also constitutes “malfeasance.”

Three key points of the law include:

§ 14-59-117 (a) (1) If the Division of Legislative Audit determines that a municipal treasurer is not substantially complying with this chapter, the division shall report the findings to the Legislative Joint Auditing Committee.
§ 14-62-102 (a)(1) If the Legislative Joint Auditing Committee concludes the process under § 14-59-117 on a municipal corporation, and in the immediately subsequent three-year period the Legislative Joint Auditing Committee concludes the process a second time, the Legislative Joint Auditing Committee may notify the Attorney General and the Governor of its actions.
§ 14-62-102 (b) Upon a finding that the conditions under subsection (a) of this section have been met, the circuit court of the Sixth Judicial Circuit shall revoke the charter of a municipal corporation under this section...

Losing one’s city charter is serious. And now ingrained within Arkansas law, a city must make it part of its accounting best practices to take information technology seriously.

2. While the law applies to application controls, it’s wise to follow all IT best practices recommended by the Arkansas Legislative Audit.

Specifically, the new law applies to application controls listed in the Arkansas Legislative Audit best practices. According to the Arkansas Legislative Audit, application controls “relate to the transactions and data for each computer-based automation system; they are, therefore, specific to each application. Application controls are designed to ensure the completeness and accuracy of accounting records and the validity of entries made.”

Application controls include areas such as data input, data processing, data output, and application-level general controls. However, it will help a city if they follow all the IT best practices listed in their document—including areas such as information systems management, contract / vendor management, network security, wireless networking security, physical access security, logical access security, and disaster recovery / business continuity.

That’s because general IT best practices create the foundation for your application systems technology. Without following general IT best practices, you are likely to create too much risk with your applications. Indirectly, you may find yourself in non-compliance with application controls if you don’t plan, invest, and proactively manage your general information technology.

3. Other states should see Arkansas as a sign of what’s to come—and prepare.

The trend for technology-related security, privacy, and best practices legislation is more, not less. Information technology now holds the crucial role of keeping citizen data private and ensuring that government remains operational even during or after a disaster.

Because government entities—including cities—often don’t spend money on implementing IT best practices even when the danger signs are obvious, laws are getting increasingly passed to ensure accountability and compliance. After Arkansas, it’s likely that other states will pass similar or parallel forms of legislation that hold local governments accountable.

In other words, if you’re not a city in Arkansas then that doesn’t mean you should rest on your laurels. Hold yourself accountable to your citizens and city operations proactively—before your state passes stricter laws like in Arkansas.

Concerned about the state of your information security or compliance with the law? Reach out to us today.

Tuesday, May 9, 2017
Victoria Boyko, Software Development Consultant
Victoria Boyko

Yes, you read our headline correctly. The Associated Press recently reported on the city of Springfield, Florida’s old website taken over by a Japanese pornographer. If citizens checked the old website URL for a period of time, then they would have viewed the homepage of a pornographic website.

The article noted:

Springfield switched its website to a .gov domain about three years ago. The city's information technology department is seeking to buy back the old domain and any domains [sic] names similar to the city's current website, springfield.fl.gov.

Before you laugh or judge, consider your city. It’s likely you already have or will do a website redesign in the future. And it’s likely you are switching or have switched from an old URL (such as a .org URL) to a new URL (such as a .gov URL).

In other words, you could make the same mistake.

Whenever you redesign and/or assign new URLs to your website, we recommend having website professionals manage all of the parts and pieces as you go from one URL to another. But two basic website best practices could have prevented Springfield, Florida’s public embarrassment.

1. Keep ownership of all old city website URLs.

Do not give up ownership of these URLs. You will need them.

First, it’s good practice to own these URLs so that other people don’t buy them, use them for websites that have nothing to do with your city, and then unintentionally (or intentionally) confuse citizens. Think of it like a celebrity or a well-known brand buying up URLs that might contains names that people would sensibly search for. It’s a way to make sure that common, incorrect website URL searches all go to your city’s website.

Second, you will need your old URLs to redirect people to your new website. It’s like a store that needs to tell people at the old location that there is a new location.

2. Use 301 redirects.

A “301 redirect” is a technical website term. It means when a person goes to your old URL they are automatically redirected to your new URL. Hubspot uses a great analogy of mail forwarding. When you move, you set up mail forwarding so that your mail goes to your new address. 301 redirects are an online version of that concept.

301 redirects are essential for a few reasons:

  • Citizens will get automatically directed to your new website when they type in your old website. Many of your citizens will not use Google to find your city’s website. They will type in or cut and paste a familiar URL—your old website. Old habits die hard, and many citizens won’t know you changed your URL. You need to make sure that when they type in that old URL they get automatically directed to your new website.
  • Other websites that link to your city’s website need to work. If 100 different websites link to your city’s website, what happens when you change your URL? All of those links won’t work—unless you use 301 redirects. This simple website tactic is the difference between people continuing to visit your website from those 100 external websites versus seeming to go dark. Without 301 redirects, it’s like moving and then not telling your friends and family your new address.
  • Your own internal website links need to work. If you do a thorough, meticulous job of changing every single hyperlink on your website to your new website URLs, then this won’t be a problem. But if you have hundreds of hyperlinks on your website that direct people to other pages on your website, then it’s likely you won’t have the time to change every link. For the time being, 301 redirects will provide a band-aid until you can change all internal links.
  • 301 redirects help you stay visible to search engines. Search engines spend years and years getting to “know” your website through indexing your site on a regular basis, analyzing links to and from your website, and detecting how much content you produce. If you produce a new website with a new URL with no connection at all to your old website, then search engines will have to get to “know” your new website from scratch. Then it may take a long, long time before you start appearing in search engine results again. For example, if a citizen types in your city’s name into a search engine, your website may not even show up in the first 10-20 search results. 301 redirects smooth things over—essentially letting search engines know that you’ve moved and keeping your city high in the search results.

While just two simple best practices, these are technical aspects of your website that need to be managed and overseen by experienced website professionals. Otherwise, like Springfield, Florida, you may end up losing ownership of your old URL, failing to redirect citizens to your new website, and then letting them think that your city’s website apparently has switched over to providing porn.

Want to ensure that you prevent a similar disaster? Reach out to us today.

Tuesday, May 2, 2017
Brian Ocfemia, Technical Account Manager
Brian Ocfemia

We often talk about data backup as the best remedy for a virus infection. If the worst happens and a virus takes your systems down, then you just restore an uninfected backup.

However, a recent article concerning Bingham County in East Idaho brings up an excellent question: What happens if your backup servers get infected?

The Idaho State Journal reports that “[Bingham] County information technology staff thought the virus was contained but discovered [on February 17, 2017] that one of the backup servers had become infected, knocking the entire system offline.”

Luckily, the county had some other data backups in place to mitigate damage from the ransomware virus attack. But this scenario offers a good lesson. Let’s address several technology pieces that need to be in place to prevent a virus from infecting a backup and permanently destroying your data.

We’ll assume in our discussion that a city already has some type of data backup solution along with antivirus software in place.

1. Monitoring and Alerting

It’s bad enough to get a virus. It’s worse if that virus goes undetected. Many modern viruses often mask themselves, retreat to the background, and do malicious things to your systems such as collect financial information. The longer the virus lurks in the background, the more it can spread and the more damage it can do.

Cities need proactive monitoring and alerting through a combination of automated software that tracks technology health combined with experienced IT professionals watching your systems. Part of that monitoring and alerting involves the right kind of antivirus software. We recommend enterprise-grade antivirus software that offers sophisticated monitoring tools for IT professionals to track and catch viruses.

2. Unlimited Offsite Data Backup Storage

Let’s unpack this phrase a bit.

  • Offsite data backup: In addition to backing up your data onsite, you need an offsite data backup component for worst-case scenario disasters such as tornadoes, flooding, or fires.
  • Storage (and retention): You will need to store various snapshots of your backed up data and make them available in case you need them. For example, you may need to see a snapshot of data as it looked one month ago if certain documents recently went missing. The right storage strategy allows you to maintain all versions of your files and documents while also retaining them for a set period of time.
  • Unlimited: This is key to rarely (if ever) worrying about a virus or ransomware attack. Let’s go back to the example of Bingham County and assume the ransomware virus lingered around for a long time, infecting even backup files. Some recent critical data might unfortunately get lost, but the county could still go back—as far as it wanted—to a snapshot of its data right before the infection hit.

It’s important to note that if you don’t have enough storage for a reasonable backup retention period, you may be stuck in a situation where the only files you can restore are infected ones. We recommend an unlimited offsite data backup storage service that allows you to keep your offsite backups indefinitely. Then, you can go back in time as far as you need to recover files.

3. Employee Education

The Idaho State Journal article goes on to state:

“An information technology director for a neighboring East Idaho county said emails with suspicious attachments can often cause computer systems to become infected. He said his systems manager comes across up to three such emails per week.”

Despite the best cybersecurity protection and data backup, employee education remains an essential part of your strategy. Antivirus and antispam software can help prevent access to many malicious websites and email attachments. But employees still need to learn more about what not to click on and how to spot hacking and phishing attempts.

Some things you need to talk about with employees include:

  • Browsing safely and knowing the signs of a malicious website.
  • Scrutinizing email attachments and understanding that hackers can spoof email addresses (such as an email supposedly coming from their boss).
  • Downloading unnecessary or unauthorized software from untrustworthy sites (such as games, shopping apps, and productivity apps).

As we see from this situation, there’s more to backing up data than just backing up data. You need to stay vigilant through proactive monitoring and alerting. You need to retain data snapshots that go far back in case your backups get infected. And you need to keep training employees who often unknowingly take actions that let in viruses and hackers.

Worried about what would happen to you if a ransomware virus hit? Reach out to us today.

Tuesday, April 25, 2017
Nathan Eisner, COO
Nathan Eisner

In past blog posts, we’ve talked about the importance of data backup for body camera video (and other police department video). We always ask, “What happens if evidence is permanently lost?”

This situation recently arose with the Cockrill Hill Police Department in Dallas, Texas. After a ransomware attack on the police department’s servers, the municipality permanently lost data. According to a Mother Jones article (with my added emphasis in bold):

The police department claimed that they still had paper copies of all the documents on the server and physical copies of much of the video. But in a letter sent to the county prosecutor, the department said "all bodycam video, some photos, some in-car video, and some police department surveillance video were lost." The department tried to recover as much as possible but said that "if requests are made for said material and it has been lost, there is no chance of recovery or producing the material."

The article opens by telling the story of a defense lawyer working for a client who faced prison time. He needed specific evidence to help his client avoid jail time. That evidence? Permanently lost. In other words, the data loss—rooted in a technology problem—could literally send a person to jail or serve a longer prison sentence because important evidence disappeared forever.

What this Mother Jones article doesn’t address is how easily this loss of data could have been prevented.

The Best Weapon Against Ransomware? Data Backup

While you may protect yourself against ransomware in many ways, the worst scenario may still happen. An employee clicks on a malicious email. Hackers break into a server. Lack of up-to-date patches expose your software to a major security flaw. It happens.

Like insurance, your technology needs to prepare for the worst. Only data backup can fully “insure” you against ransomware. Let’s say the worst happens. Ransomware is downloaded, you receive an automated blackmail threat, and you (wisely) decide not to pay the criminals. You permanently lose that data. But luckily, you have a backup you can restore. You may end up losing none of the data, as little as only minutes of data, or, at worst, hours or days of data.

For data backup, you need:

  • Onsite backup that takes frequent snapshots of your data. For smaller disasters (like files lost or a server failure), you can recover quickly.
  • Offsite backup that sends your data to a geographically distant data center (or centers). Then when a disaster wipes out your onsite data, you still have all your data safely stored offsite.
  • Regular testing (such as quarterly) so that you know your data backup works. Too many cities never test their data backup and they often find it doesn’t work when a disaster actually hits.

Because body camera, dashcam, and other police video requires massive amounts of video storage, it’s wise to explore data backup solutions with unlimited offsite storage. You don’t want to lose data arbitrarily because of storage caps or added costs. Unlimited offsite storage also gives you flexibility with data archiving and retention to help you follow the law.

The Second-Best Weapon Against Ransomware? Proactive IT Monitoring and Management

On the preventative side, it’s essential for police departments to hire staff or a vendor that proactively monitors and maintains technology for servers, desktops, and mobile devices. That includes:

  • Shoring up cybersecurity weak points in your network through locking down and properly configuring your computers, servers, switches, routers, and firewalls.
  • Monitoring your technology’s performance and health 24x7x365 and receiving alerts about problems.
  • Using antivirus, antispam, and content filtering software to help employees with safe internet browsing and email.
  • Consistently applying updates and patches to your software.
  • Ensuring any remote access is secure when teleworking.
  • Managing and tracking all technology assets.

While articles like the one we’ve referenced from Mother Jones seem to indicate that failure can be shrugged off without consequences, that may soon no longer be the case. Federal and state laws and regulations increasingly push for higher cybersecurity accountability from government entities. Even at best, these incidents are an embarrassment for cities and, from an ethical perspective, negatively impacting the lives of defendants (especially if they’re innocent of a crime), defense attorneys, and prosecutors who rely on this evidence to uphold the law.

Would your city’s police department survive a ransomware attack? Reach out to us today if you’ve got any doubts.

Wednesday, April 19, 2017
Mike Smith, Network Infrastructure Consultant
Mike Smith

Licking County, a county east of Columbus, Ohio, recently experienced a bad ransomware attack on its IT systems. Ransomware is a specialized virus that encrypts files—making them nearly impossible to access unless you pay criminals a ransom. Cybercriminals use ransomware to extort money in return for unlocking your files. Many organizations pay the ransom despite the FBI and other law enforcement agencies recommending against it.

Luckily, Licking County managed to mostly survive the attack based on implementing some important best practices. Let’s look at the good, bad, and ugly of this situation to extract some important lessons.

The Good

Data backups

The difference between getting crippled and devastated by a ransomware attack versus surviving it relatively unscathed all comes down to data backups. Licking County ended up losing only about one day’s worth of data for most systems. Another county referenced in the article ended up paying a ransom of $2,500 to cybercriminals because they did not invest in data backup.

Activating a plan to shut down the network

To stop the spread of the ransomware, Licking County shut down its network. Clearly, the county had a plan in place and enacted it when the ransomware virus hit. By planning ahead, they were best prepared for what to do to keep the virus contained and to minimize impact.

Rebuilding systems based on highest priority data

As part of its disaster recovery plan, the county rebuilt its systems based on the highest priority data first. The article references data such as “servers that house felony-case tracking for the prosecutor's office and the auditor's property-records database.” Any disaster recovery plan needs to have a clear plan as to how data will be restored—and in what order of priority.

The Bad

Rebuilding systems will take a lot of time

Licking County is a big county and so it needs to reformat about 1,000 computers as part of its rebuild. That takes a lot of time. Even smaller organizations will need to spend significant time rebuilding servers and reformatting computers.

Direct and indirect costs

Directly, the costs of billable IT time and possibly enhancing networking equipment and cyber protection software can present a big hit to your budget. Indirectly, lost productivity wastes expensive employee salaries and potentially delays major projects when time is ticking.

Impacts to citizen service

After a disaster, a crippled government entity will not be able to serve citizens at full capacity. The mission of government gets impacted when ransomware hits. County Commissioner Tim Bubb says, “We have lost a large part of our focus on serving the people of Licking County. What price do you put on that?"

Potentially weak firewall and network connections

A Columbus Dispatch article mentions that the county needs to shore up its “firewall and network connections.” An improperly configured firewall can leave ports open that allow hackers to easily gain access to servers and steal information. Setup of switches, routers, and other networking equipment also impacts security.

Potentially weak passwords

The same article mentions that the county needs to encourage employees to change passwords more frequently. In a recent blog post, we said, “The longer a password is in use, the more likely that hackers will be able to crack it. The more you change passwords, the more difficult you make a hacker’s job.”

The Ugly

911 dispatching affected

An article published in the Newark Advocate the day after the incident stated “...the 911 Center has been operating in manual mode since late Tuesday night. The 911 Center phones and radios work, but dispatchers do not have access to their computers. The public can still call 911 for emergency police, fire or medical response.”

While not completely shut down, any impact to 911 or other critical emergency services can literally affect lives in the wake of a ransomware attack.

Employees click on too many suspicious emails

One of the biggest cybersecurity threats is people. No matter how great your data backups, antivirus, firewalls, and security measures, hackers and cybercriminals still often break into a government entity through people clicking on suspicious websites and email attachments.

Note this paragraph in the Columbus Dispatch story:

Fairfield County started working last year to tighten procedures to guard against the type of cyberattack that occurred in Licking County, said Fairfield County IT Administrator Randy Carter. He said he was dismayed when he sent a test phishing email to county employees in September and more than 25 percent clicked on it. Carter plans to provide training to employees on what emails to avoid.

25 percent! One in four people got fooled by these dangerous emails. Each click on one of these emails opens you up to the threat of a virus or ransomware.

Cybercriminals targeting government more and more

Cyberattacks grow more numerous and targeted. Government entities are ripe for these attacks. That includes cities.

Are you prepared?

  • Like Licking County, do you have data backups to recover from a ransomware attack?
  • Do you have the right network equipment and modernized technology to protect yourself?
  • Are your employees trained about the dangers of clicking on malicious emails and websites?

If you need help protecting yourself from a ransomware attack, reach out to us today.

Tuesday, April 11, 2017
Brandon Bell, Network Infrastructure Consultant
Brandon Bell

A city had operated for a long time with tape backup and decided to upgrade. City administrators heard from their IT staff that they needed something more reliable than a manual solution reliant upon busy people to both conduct the backup and store it offsite.

Spending a lot of money on a modern complex data backup solution, the city was assured by its IT staff that this automated beast would solve all their problems. Indeed, the data backup worked automatically. In a meeting, IT staff showed city department heads the wonder of the data backup system by retrieving a few PDF documents from the backup data storage. To city council and the public, the city administrator proudly said they had ticked data backup off their list. Problem solved!

One day, a fire tragically tore through most of city hall. The building ruined, city staff needed to relocate to a temporary building until a new city hall was built. But thank goodness—despite all the servers destroyed—that the city could retrieve its data.

Or not. When IT staff attempted to restore the city’s data through its backup, most of the major databases, applications, and data would not restore. A few chunks of data—like some people’s individual documents—were okay. But the city’s most important information was not there.

And so...an expensive backup solution became nearly worthless. Why? Upon further investigation, the city administrator was told that nobody ever tested the data backup. “But...it was an expensive solution,” the city administrator said. “And my IT staff said that it was automated. The data backup solution’s reporting even said it worked.”

Well...it didn’t. And that’s all that mattered when the city administrator had to now explain why this expensive investment failed them after a disaster—and failed to do the exact thing it was supposed to do.

Preventing This Disaster

One aspect of data backup and disaster recovery—testing—is nearly as crucial as simply having data backup at all. No matter what kind of data backup you use, you need to test it. Otherwise, you don’t know that it’s working.

Let’s look more closely at the errors in our city scenario above.

Error #1: Assuming the data backup works.

A data backup solution will often look like it’s doing its job. From manual solutions like tape to more sophisticated automated data backup servers, the data backup application will often indicate that the process is a success or failure. But no matter what the application tells you, you don’t know that it works until you test it.

Error #2: Not testing all the backed up data.

Calling up a few files such as PDFs from the data backup storage is not testing. When a disaster hits, you will need to be fully operational with your databases, software applications, website, email, and documents. For example, will your account system work from a backup copy? When you test, test everything. Simulate what would happen if an actual disaster hit.

Error #3: Develop a plan and document it.

Testing needs to be part of your overall disaster recovery and business continuity plan. The act of testing not only guarantees you will access the data but also allows you to practice how data recovery will work. Who does what? How fast will the data be restored? In what order? Where will you access the recovered data?

You want to run into issues during testing and deal with them in a simulation—rather than after a real disaster.

Uncertain about your data backup solution? Are you testing it at least quarterly? Reach out to us today.

Tuesday, April 4, 2017
Jabari Massey, Network Infrastructure Consultant
Jabari Massey

On the surface, a coastal city did some correct things to back up its data. The city had a few servers in a physically secure basement room that were well-maintained by IT staff. One of the servers backed up important data. In case a server failed, the backup server would run until the city could replace the original server.

A long time had passed since the city last experienced a hurricane. When a hurricane finally seemed eminent, the city was ordered to evacuate until the massive storm passed. The city manager and IT staff didn’t think much about the servers other than placing them upon concrete blocks in case of flooding. As long as the city implemented its emergency action plan and evacuated everyone safely, the city manager assumed its information technology would remain safe.

After the hurricane passed, city staff returned to find that no massive devastation occurred but they did experience heavy flooding. The IT staff had placed the servers upon concrete blocks as a precautionary measure, but they learned an incredibly hard lesson in hindsight.

Located in a basement room, the servers sat below sea level. Although the rest of city hall experienced moderate flood damage in places, the basement had filled up to dangerously high levels. All of the servers—including the backup server—were rendered unusable by the flooding.

With a sinking feeling, the city manager realized all critical data—including financial, public safety, document management, email, and website data—was gone. The only backup server got destroyed along with the others. It might be easy for the city manager to point some blame in the direction of the IT staff, but it was well-known that he had refused requests to explore other data backup options because of “budget concerns.”

Now, the mayor, city council, the media, and public would be asking questions.

Preventing This Disaster

Sure, the city manager and IT staff made a bad decision to place servers in a basement room below sea level. But their errors go deeper than this poor choice of physical location for the servers.

Let’s look at the errors in the story above.

Error #1: Locating servers in a flood-prone area of your building.

Getting the most obvious error out of the way, it’s clear that the servers needed to reside on an upper floor. In addition, the server room needed to be in a room that mitigates flood risks through preventative measures such as water leak sensors or eliminating areas where water can enter.

Error #2: Lack of offsite data backup.

While locating the servers on a higher floor may have prevented this immediate flooding disaster, it’s still not a full disaster recovery plan. Anything can happen to your technology onsite. To guarantee full recovery of your data after a disaster, you need an offsite data backup component to your emergency plan.

We recommend storing your data offsite in geographically dispersed locations (such as in data centers both on the East and West coasts). Then, even if the worst disaster wipes out your buildings, you will be able to recover and access your data.

Error #3: Lack of technology planning.

The lack of offsite data backup also signifies a larger issue—a lack of planning. The city had developed an emergency plan and used it in the case of the hurricane. But when was the plan developed? When was it last updated? Did it include technology-related scenarios? What was the plan to protect data in case of a disaster?

First, the city needed to update its emergency plan and include technology. That would have addressed technology-related gaps in the city’s data backup, disaster recovery, and business continuity plans. Second, the city needed regular technology planning meetings (at least once a quarter) and ongoing monitoring to ensure that data backups were tested and working. This regular monitoring and planning would help the city adapt to changes (such as new technology, more staff, building changes. etc.) and ensure that the risk of data loss is minimal.

Flooding is one of the most common disasters. It can happen anywhere in the country and devastate a city. Because citizens will rely on your city after severe flooding, you must be operational as fast as possible. That means having access to your data—your website, your documents, and your applications that are essential to operations.

By developing a disaster recovery plan that includes an offsite data backup component, you will lessen the risk of permanent data loss and angry “Why?” or “How?” questions after the fact from council, the public, and others.

Concerned about your data backup and disaster recovery? Reach out to us today.

| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 |