Blog Host: David Sengupta - Global Director of Product Management, Quest Software READ ENTIRE BIO

Sengupta's best practices for Microsoft Exchange & compliance

1. Be proactive about Exchange & compliance
   Don't ignore or defer compliance -- ignorance is NOT a valid defense. Neither is size of your organization or sheer volume of data involved. Neither is complexity or cost associated with becoming compliant. Your job could depend on it. (So could the next 25 years of your executives' lives, for that matter.)
2. Consider all types of Exchange compliance (not just regulatory compliance)
   You need to have a plan addressing all three types of compliance, namely (i) compliance with internal company policies, (ii) regulatory compliance, and (iii) legal compliance.
3. Treat Exchange compliance wholistically
   Whether you're providing searchability, establishing controls, logging access, or maintaining integrity, all aspects of Exchange compliance must be addressed systematically as you assemble an end-to-end solution for your company. All storage "silos" where e-mail may exist within your organization (online, backups, file system/PSTs/mobile devices and archives) must be addressed if you're going to be comprehensive in your approach. An archive is only a small part of the solution. Backups are for disaster recovery and provide some value when reacting to compliance investigations, but are neither intended nor adequate for compliance purposes.

   If you compare becoming compliant to building a car, it's fine having a running engine, but without wheels you're not going to get too far (or it may be way too expensive to do so by foot, carrying the engine. In the end you may miss opportunities, risk your reputation, and be unable to meet certain requirements -- often with significant penalties -- as a result).

4. Ensure searchability across all Exchange storage locations
   As a bottom line, compliance is all about maintaining records (e-mails in this case) and being able to discover, recover and produce them in a reasonable amount of time if needed. This implies (a) that the data is retained in accordance to whatever policy, regulation or law is applicable, (b) that you know where the data can be found, (c) that you can find it rapidly and (d) that you can produce and deliver it to whatever requesting party needs it.
5. Manage e-mail as a record
   Regardless of what people say, e-mail provides a record of communications, facts or events within your company, and therefore needs to be treated as a record. You need to consciously make sure you're storing and addressing all the parts of an e-mail, namely P1 headers to prove authenticity, P2 headers, subject lines, message bodies, attachment metadata (noteably filenames and date stamps) and attachment contents.
6. Establish e-mail controls
   E-mail controls must be established for every one of the storage "silos" (online, backups, file system/PSTs/mobile devices and archives). Any access to e-mail -- wherever that e-mail resides and in whatever format it exists -- must be secured and access logged if you are to ensure security, traceability and ultimately auditability. Logs themselves must be managed, and you need to be able to ensure that an ambitious rogue administrator (for example) doesn't go altering the logs themselves in an effort to cover their tracks. And the same applies to the e-mail records themselves -- you need to ensure they have not been altered from their original form by someone attempting to cover their tracks.
7. Don't forget about destruction
   While I haven't addressed retention and destruction policies during these two weeks, keep in mind that regulations or laws may define both how long you retain data AND after which period of elapsed time you must destroy data. Not destroying data can be as serious a violation as not retaining it. Similarly, you need to make sure you're keeping on top of stale e-mail objects -- mailboxes, public folders and distribution lists -- as these may lead to breaches of compliance.
8. Keep pace with changes in regulations
   As you've probably noticed, the pace with which new regulations and laws appear has been quite hectic over the past 18 to 24 months, with the courts and regulatory bodies trying to get on top of the many far-reaching breaches of regulation and illegal activities that have become increasingly visible across numerous industry verticals and in many countries spanning the globe. While the courts still have much work to do to catch up with the pace of technology, keeping on top of -- or at least cognizant of -- the latest developments as they impact your organization has become a requirement for virtually every organization doing business today, regardless of size, geographic location, financial status and technologies in use. For starters, consider the following regulations: Sarbanes-Oxley Act, HIPAA, SEC 240 Section 17a, FDA 21 CFR 11, Basel II, UK Freedom of Information Act, UK Data Protection Act and the Canadian Freedom of Information Act.
9. Build your solution out of people, process & technology
   In your efforts to be systematic and comprehensive in Exchange compliance, you can't forget the fact that a combination of people, process and technology are required as part of the solution. The right people -- with the right skills and expertise -- need to be in place, security cleared, trusted, and empowered to do what needs to be done. The right processes need to be in place such as e-mail policies, for instance (see my SearchExchange.com Step-by-Step Guide: E-mail compliance to-do list for further discussion of e-mail policies). And finally the right technologies need to be in place (remember, they're all still evolving and no one vendor has all the solutions) in order to handle the scalability, automation, storage, search and other elements of compliance in light of your company's infrastructure and compliance requirements.
10. Learn from others' experiences
    As a final point, look to the experiences of others -- especially where such experiences are summed up in a collective manner. Start with the best practices defined in British Standard document BS DISC PD0008 which has very relevant recommendations around compliance with the code of practice for legal admissibility, and BS DISC PD0009 which focuses on legal admissibility and evidential weight of information stored electronically. And continue to ask us questions on Exchange & compliance… and read our future compliance articles on SearchExchange.com!

Thanks
I have sincerely enjoyed the opportunity to share these two weeks with you, and look forward to continued discussion around Exchange & compliance in the weeks and months to come. If you're looking for me you'll find me lurking in the background at:

And finally, as always, you can reach me via e-mail at mailman@quest.com.

Blessings, g'day & happy complying ...

David
Posted by David Sengupta Finding e-mail evidence rapidly in your Exchange environment
14 JUL 2005 18:07 EDT (22:07, GMT)

Recap
Yesterday we talked at length about the e--mail controls that you need to establish to strive towards compliance in your Exchange environment. Those were all things that you need to do to be proactive about compliance.

Reactive Exchange compliance
Today we'll talk about your options in being reactive to a compliance--driven request. The vast majority of the Exchange--related reactive scenarios I hear about (almost daily) are driven either by lawsuits or by regulatory requirements, often SEC--initiated. Obviously internally--driven investigations tend not to be widely publicized and don't have the high--visibility penalties and loss of reputation associated with them, but they are significant as well. The scenarios which make it to my Inbox often affect large globally--distributed environments and frequently involve having to search vast volumes of data in order to find that proverbial needle--in--a--haystack ---- the e--mail evidence required to defend against some allegation of legal or regulatory non--compliance, or to prove violation by a third party.

Options
The following table will address options to respond to a compliance investigation if you have not established the controls we recommended yesterday:

E--mail storage "silo"	Reactive Options to Find Evidence in your Exchange Environment
Overall	Before we start drilling into the various 'silos', I should say that there are three over--arching options I typically see companies take in the face of a compliance investigation. Hire a Consultant Oftentimes I hear of companies who opt to bring a consultant in--house (or in some cases to work remotely) to discover e--mail evidence and prepare it for trial. Any number of options exist here, from simply searching the network for any e--mails within Exchange online data and PSTs, all the way through to processing the resultant evidence (which can consist of millions of e--mails), de--duplicating (duplicates and revisions ---- especially of attachments ---- are common throughout e--mail), Bates stamping (labels by the courts, if applicable), and conversion to PDF, TIFF, or whatever format is required for the production of evidence to the requesting party. Most of the major consulting firms offer these types of services, typically through their legal technology teams, data forensics teams, compliance teams, or similar. Needless to say, costs are definitely a factor. Focus Attention on Backup & Outsource Recovery While I'm not a lawyer and can't provide legal advice to what is admissible and what isn't, I do see many companies that focus their efforts on backup media, oftentimes shipping literally thousands or tens of thousands of tapes offsite to 'outsourced recovery providers' for recovery and subsequent hosting of recovered e--mail in some remotely--accessible secure digital archive. The services provided by these outsourcing companies can range from EDB--to--PST, EDB--to--MSG or EDB--to--TXT data conversions, through full de--duping and other services as discussed in the previous section. Needless to say, large volumes of outsourced data processing and hosting like this can easily run into the millions of dollars and take weeks or months of around--the--clock recovery to perform. Due to the lucrative fees associated with these types of services (largely associated by the amount of manual labour required and the magnitude of the fines associated with failure to produce evidence) there are quite a number of vendors that have emerged in this space, some with labs spanning the globe focused primarily on Exchange recovery operations. Pay Up There have also been cases where for some reason or other (presumably recommendation of corporate counsel weighed against cost, complexity and risk) where companies have decided to bite the bullet and simply (sic) pay the fines associated with failure to produce evidence. When you consider that restoring, say, 50,000 backup tapes and finding all e--mails relating to a particular topic, storing the results and incurring the associated legal fees could easily run into the 10s and 100s of millions of dollars, paying a fine in the 10s or 100s of millions of dollars may not be a terrible alternative. Do it Yourself Finally, depending on scope of the investigation, time pressures, magnitude of penalties, and other options, more and more companies are taking the bull by the horns and attempting recovery and discovery options in--house. The remainder of this table will cover different options for performing in--house recovery and discovery. Scalability Regardless of what option you choose, challenges of scale will almost always come up, since the magnitude of data that you'll be searching is almost definitely far greater than what you're used to dealing with. Temporary storage locations will likely be required (i.e., for EDBs and PSTs) and the amount of temporary storage needed could easily be in the 10s or 100s of terabytes, or more, if you're at a medium-- or large--sized company.
Online data	Searching production mailboxes is challenging, and not trivial when you have a large, geographically--distributed Exchange organization. Options include: ExMerge The Microsoft ExMerge utility can be run against online mailboxes in order to extract data meeting certain criteria into a series of mailboxes, one per mailbox. Limitations are that ExMerge cannot run against multiple servers simultaneously. Benefits are that ExMerge is free. Third--party Reporting Solutions Numerous third--party Exchange reporting solutions exist, all with different capabilities. Some permit search, audit and/or reporting against senders and recipients. Some support subject line auditing. Some support reporting on data sent to/from other SMTP domains. Others support reporting on content and analysis of attachment types. And other support taking action on production content, in order to export to PST or otherwise. Manual Mailbox Search Occasionally for smaller investigations I still hear of the 'traditional' approach to searching a mailbox (typically for HR--driven or similar purposes) – simply giving an account permissions to log onto the mailbox, defining a MAPI profile, and logging into the mailbox (if you do this, watch out for those "Welcome" messages that Outlook generates whenever you log into a new MAPI account … they can alert end users that someone is watching over their shoulder!) Challenges The biggest challenges with searching online data are (i) not impacting production Exchange server usage during a search (many searches are MAPI--based and therefore impact server performance while they're running), and (ii) getting 'close enough' to the server to have sufficient network performance and to minimize network impact. I have seen some customers make the decision to forego online production search and take a backup of the server instead, then run search and analysis against the backed up EDB file instead of against their production server.
Backup data	Searching Exchange backups has challenges of it's own, and once again is not trivial if you're in any mid-- to large--sized company. Vendor and Version Support When you think through your Exchange server backups going back several years (as far back as you retain) you'll often find that these span numerous backup solutions from different vendors, and often different versions of backup software. They probably also span different versions and service packs of Exchange, and may even be comprised of a combination of full, differential, incremental, snapshot and other backup types. Just figuring out what software you'll need could be a problem, and one of the first things you'll want to do is figure out what software and what versions are entailed, if you don't already track this information. Recovery Environments A few options exist for do--it--youselfer. The most obvious one is simply (sic) restoring each backup sequentially (or in parallel if you have the resources), then building a recovery environment for each (unless you have brick--level backup software), and searching through each recovery server with ExMerge or similar in order to extract evidence of interest. Recovery Software If building (and maintaining) a recovery environment is too resource intensive and too slow for you, then you may want to consider third party recovery software. Two vendors provide solutions in this area, namely Quest Recovery Manager for Exchange (disclaimer – I work for Quest) and OnTrack PowerControls. These products provide varying levels of search functionality – with Quest providing support for search within attachments and more granular search – and varying levels of support for 3rd party backup integration. Bottom line, both these solutions circumvent the need for a recovery environment and can speed the process of getting into the evidence you're looking for. Backup--to--Archive Migration One option I am seeing occasionally is companies choosing, is to spend the additional money up front – now that they're faced with a compliance--driven investigation and have to search their backups anyways – to deploy an archival solution (i.e., Quest, Symantec (formerly Veritas (formerly KVS)), OpenText (formerly IXOS), Zantaz (formerly Educom), etc.) and migrate their backups into the archive. Depending on whether a recovery environment or recovery software is used, this can add substantial amounts of labour costs to your recovery, but the end result is that all (or whatever subset you choose i.e., the last 2 years)) your historical Exchange data is now in a searchable archive to minimize future costs.
Offline data in the file system	When it comes to offline data in the file system, I really only hear of companies addressing the PST component (so far). PST--to--Archive Migration Similar to the backup--to--archive migration we just discussed, I'm seeing customers frequently address searching their PSTs by deploying an archive first, then migrating all PSTs into the archive, blocking or limiting subsequent new PST creation (technically or by written policy), and then searching the data once it's in the archive. This is definitely a recommended approach, though necessitates an archival solution. In--place PST Search I have heard of occasional cases where the urgency of a case, the lack of an archive, or the limitations of sufficient temporary storage space warrant searching the PSTs in an environment in--place. C2C ActiveFolders is an example of a product that claims this kind of solution. Desktop Search Tools Much of the spotlight lately has been on Google Desktop Search, MSN Lookout and MSN Search, specifically in how they simplify finding e--mails and/or files across local and network locations. While none of these tools is ideally suited for enterprise search spanning PSTs across an environment, they do have varying levels of ability to search UNC--based shares and therefore index remote locations, including PST content. Much work is still required and the challenges of scale and network bandwidth are substantial, but it will be interesting to watch this space evolve over the coming five to ten years. As yet I have not heard of any vendors who support search across mobile devices, Blackberries, or other offline locations, however we're seeing technologies such as mobile wipe evolve to at least provide some level of control over the data in these locations. It will be interesting to see how far search makes its way into these devices.
Archive data	Searching the Archive In the case that you've deployed an archive, you're probably in good shape. Archival solutions ---- especially those centered around compliance ---- come with varying levels of categorization and search, so finding evidence in the archive should be simple.

A Word about Privacy
Well, here I go again, I've written another long blog. Well before we wrap up this, our second--last day of the Exchange Compliance Expert Answer Center, I should remind you of your obligations around privacy. Most countries have privacy legislation that defines what you can and cannot do with regards to reading your staff e--mail, and if in doubt, you're always advised to err on the side of caution. This is especially true in countries such as France and Germany, but regardless of where you are located, you should NEVER read someone else's e--mail without the proper motivation, justification, authority and documentation. Always leave a paper trail and always ensure you have written (or e--mail  ) management approval and/or direction prior to engaging in an investigation. Anything less can get you into trouble, or cost you your job.

Well this rounds out our two--day discussion of being proactive vs. reactive to Exchange compliance. I trust it has been valuable to you so far, and that if anything, I have gotten you thinking about Exchange & compliance. One last day to this particular Expert Answer Center so tomorrow we'll wrap things up and call it a wrap.

Have a great day!

David
Posted by David Sengupta E-mail controls & Microsoft Exchange
13 JUL 2005 15:40 EDT (19:40, GMT)
2005-06-12Recap
Yesterday we rounded out our discussion of the parts of an e-mail, so we have now addressed the "who, what, where and why" of Exchange compliance. Today we'll shift our focus to the "how."

Proactive Exchange compliance
I hope that you all had a chance to do your "homework" and review my "E-mail Compliance to-do list" on SearchExchange.com as I recommended yesterday. Today's blog will be a long one as I will try to address rapidly the various e-mail controls that you'll need to establish to ensure access to data is secure from a compliance perspective. Let's dive right in.

Establish e-mail controls for Microsoft Exchange compliance
The following table reviews each of the 'silos' and indicates the controls you'll need to consider establishing in order to minimize risk of a breach of compliance:

E-mail storage "silo"	E-mail controls for Microsoft Exchange compliance
Online data	Mailbox security Ponder this question for a moment: Who in your organization has been reading through your Inbox today? The bottom line is that you need to be able to clearly identify everyone who has permissions to any mailbox within your Exchange environment. You also need to be able to examine any account and rapidly identify the mailboxes that this account has access to. And ideally, you have a mechanism in place (i.e., auditing & event logs) that tracks all security changes to any mailbox and retains this in an easily accessible but non-tamperable audit history. As a minimum if you don't have an audit trail, you should at least have a clear up-to-date list of sensitive mailboxes in your Exchange organization and run regular security audits on these mailboxes. Public folder security Very similar to mailboxes, you need to be able to identify accounts with rights to any public folder in your environment, and also be able to look at any public folder and rapidly determine who has access the public folder. Knowing which public folders contain sensitive data is important, and using consistent public folder naming conventions would really help in this area (i.e., all financial data is in folders starting with "FIN," etc.). Again, having some mechanism to audit all security changes on your public folder hierarchy is ideal, or as a minimum knowing which public folders are sensitive and periodically auditing to ensure no one has been granted elevated permissions is recommended. Distribution list membership Often overlooked, distribution list membership is critical, as membership in a distribution list essentially implies access to all of the documents and e-mails distributed to that distribution list. You need to be able to quickly report on membership, or conversely, take any mailbox and determine rapidly which distribution lists that mailbox is a part of. Once again, auditing changes to distribution list membership is ideal, or as a minimum knowing which lists are sensitive and periodically auditing them is recommended.
Backup data	Access Bottom line with backup data is controlling who has access to your backup media. This includes the following: Controlling physical access to any and all rooms where backup media are stored (including actual tape drives and tape libraries in your datacenter and branch offices) Maintaining and managing passwords for backup/recovery Controlling physical access to all backup media in transit between remote sites and hub sites, and as they are transported to offsite storage locations Controlling physical access to backup media when shipped for compliance investigational purposes
Offline data in the file system	PST usage & security Key controls here include: If PSTs are in use, then assume they can be opened regardless of whether they are password-protected (they can be) Ensure that all PSTs anywhere on your network are secured. This includes laptops, network file shares and desktop PCs Think through liability and risk associated with loss of a laptop containing PSTs. For example, how would you address the loss of a laptop containing 10 PSTs with 200,000 corporate e-mails on it? Do you have controls in place to mitigate this risk? PST usage should be minimized or eliminated wherever possible OSTs OSTs should also be secured. Contrary to popular belief, OSTs can be opened by someone with sufficient information at their disposal. Mobile devices Wireless data on BlackBerries and other devices should be protected through any of (i) password, (ii) remote wipe capabilities, (iii) no local storage, etc. E-mails in the file system E-mails saved to .MSG or .TXT files should be in secure locations.
Archive data	Journaling Typically, most archival solutions employ Exchange journaling in order to bifurcate (duplicate & fork) messages on the transport path and archive them. When we spoke of header data the other day, you'll remember the importance of P1 header information. Bottom line is that there are different types of Exchange journaling, including message journaling and envelope journaling. Only envelope journaling captures P1 header information -- specifically BCC recipient addresses and distribution list membership at the time of expansion -- critical to having a complete picture on what actually happened to a message and who actually received it. What's inside An archive is only as good as what's inside the archive. In other words, you are still responsible for all the Exchange data that exists from before Day 1 of your archive deployment. Access Who has access to your e-mail archive (if you have one)? Control to archives must be strictly enforced and logged. Retention Ensure that you are using some sort of categorization mechanism to automatically apply the correct retention policies to e-mails wherever possible. The volumes of data entailed in a corporate e-mail archive are massive, and relying solely on manual procedures is not typically going to be sufficient for compliance purposes.
Overall	Audit history As a general guideline, you need to think through each time anyone has access of any sort to read an e-mail -- be it using Outlook, browsing an archive, searching a backup tape, etc. -- and make sure that you work towards auditing every step of the process. Many of the procedures and technologies to achieve this end-to-end are still evolving as we speak, but in an ideal world you need to be able to report very quickly on anyone who has ever read a particular e-mail anywhere -- in a PST, in a mailbox, in a public folder, off a backup tape, in an archive, or on a mobile device -- from a compliance purpose. Needless to say we're not there yet, but that's part of the holy grail of e-mail compliance. Administrative access Going hand in hand with the previous point, you also need to think through anyone who has administrative access to any of the areas mentioned. If a rogue administrator decides to "give themselves access" to someone's mailbox or the file share where a PST is kept, or add themselves "temporarily" to a confidential distribution list, or if they recover a backup or go trolling in your archive, would you know? You need to establish controls to minimize or eliminate the possibility that this will happen, and obviously have audits in place that will track all of these activities if they do occur. Stale objects Stale objects, such as mailboxes, public folders, distribution lists or accounts, all pose risk to your environment. You need to have a mechanism to identify stale objects, then a procedure to delete them as part of a de-provisioning process in your organization. Integrity of Data Finally, you need to establish controls -- be they unchangeable media or otherwise -- to ensure that the audit and archive data that essentially are "corporate records" are actually stored in an untamperable way. You need to have the confidence that whatever audit history or record of e-mail communications you're looking at (or taking into court!) is indeed a true representation of the actual events that happened. Enough said.

People, process & technology
So like I said, this was going to be a long blog. I've spent some time addressing the various e-mail controls that need to be implemented to ensure compliance in your Exchange environments. Now before you all pick up the phone and call your local software vendor looking for a product to do all of what I said (NONE do… and if they claim to, or position their entire companies around "compliance" -- the "fad of the day" -- be wary) … you need to keep in mind that a combination of people, process and technology are required to devise a solution to the compliance "problem" as it pertains to your Exchange organization. And don't forget scalability -- whatever solution you end up devising needs to be able to process relatively massive amounts of data. Scalability is by far the most common bottleneck I hear of from companies deploying software as part of their compliance solution.

I just can't overemphasize the fact that no one software or hardware product is sufficient to address all of this. No one person can guarantee compliance in your organization. No one written e-mail policy -- even if signed by all your staff -- can ensure appropriate and compliant usage of your e-mail system. And to be honest, no-one in the industry has thought through every possible angle to compliance and Exchange. Commpliance is like fighting SPAM; things are always evolving, always changing. Laws are being established and tested in the courts… executives and entrepreneurs are thinking of new ways to try to get rich quick through any number of creative ways… new laws are being written to clarify what's legal and what's not… and new features and functionalities in Exchange Server, Outlook and Windows -- as they evolve from release to release -- are constantly broadening and simplifying our ability to establish e-mail controls as we grapple with the volumes of data involved in Exchange environments and strive towards that elusive yet mandatory goal of compliance.

Have a great day!

David
Posted by David Sengupta What's in a message? (part three)
12 JUL 2005 15:07 EDT (19:07, GMT)
Recap
Last blog we talked about the importance of what's in a subject line, and how that can tell us something from a compliance perspective. We saw that aside from simply being an indicator of the content of an e-mail, a subject line can also provide indicators of read status and other system information of value to the compliance investigator trying to put together the history of a particular breach.

The e-mail message body…
Today we're going to take a look at the body and attachments within a message, which is where the real "meat" is from a compliance perspective. As we're making our way through these e-mail parts (header, subject, body, attachment) you should start thinking about our next topic of consideration, the implications on controlling access to all of this data – regardless of the e-mail storage 'silo' it's in – from a compliance perspective.

Before we start, keep the following tidbits of information in mind:

Industry estimates are that approximately 60 percent of all corporate intellectual property is stored within e-mail (Source: StorageInc. Magazine. Killer App – New E-mail Requirements Are Driving Significant Technology Purchases Greg Arnette, Sept. 2003)

The typical user stores more than half his/er critical business information within the confines of the messaging system (Source: Osterman Research, Inc. "Enterprise E-mail Archiving: Market Problems, Needs and Trends. August 2003)

These numbers explain why virtually every corporate lawsuit or compliance-related investigation today ends up looking at e-mail evidence. The vast majority of that evidence is found either in the message body or attachment.

E-mail message part

Importance to compliance

Body

Often ad hoc The message body of an e-mail is frequently brief and often written without much thought. As a result, users often display raw emotion or say things they would normally not say in other forms of communication. Consider the following. In a 2003 study on e-mail and organizational productivity, T. W. Jackson found that: 70% of e-mails were opened within six seconds of receiving 85% of e-mails were opened within two minutes of receiving (Source: Jackson, T.W. et al. Understanding E-mail Interaction Increases Organizational Productivity. Commun. ACM, Vol. 46, No. 8 (August 2003), pp. 80-84.) From an investigational perspective, then, there are often clues within the body of a message that might not appear elsewhere in e-mail. To some extent, much of what is found within the body of an e-mail is similar to what one might find in an Instant Messaging conversation. Standard keywords Similar to our discussion of subject lines, there are common keywords that appear in the body of a message. Some examples include: "Sent from my BlackBerry Wireless Handheld" "(sent via BlackBerry)" "Your Account" "New Password" which once again provide clues to help re-create the "big picture" underlying a compliance-driven investigations. Even something as simple as searching for FTP:// in the e-mails in a given mailbox can lead you to additional evidence in previously-unknown locations that you were not aware of.

What's in that attachment?
Probably the most important part of an e-mail from a compliance perspective is the attachment.

According to Radicati, over 85% of all e-mail data storage volume is comprised of attachments. If you allow for two-thirds of these to be duplicates and revisions of the original item (Source: Mauldin, Christopher. NewGove Solutions White Paper, Understanding the Overwhelming Need for E-mail Management, 2003) then you can really break down the enterprise Exchange e-mail storage landscape to be something along the lines of:

15% message bodies

35% unique attachments

50% duplicates or revisions of the unique attachments

Whatever perspective you take, the importance of e-mail attachments to a compliance investigation is absolutely paramount.

E-mail message part

Importance to compliance

Attachment

File types Typically the type of file attachment tells you something about its importance to a compliance investigation. JPGs and GIFs are generally of very low interest or relevance, while DOCs, PDFs and XLS files are much more interesting. Date and time stamps Both the "creation time" and "last modified time" for a particular attachment are important, especially if you are attempting to piece together a particular sequence of events. These should be taken alongside the timestamp on a particular e-mail (pay attention to GMT/UTC time versus local time offset), and (if required), correlated to remote access (VPN, other) logs to see when and from which IP a particular user connected to the network. By doing so you can build up a pretty detailed history of a suspect user's activity concerning a particular topic or during a particular timeframe. Needless to say, the massive volumes of data that you need to deal with as part of such an investigation need to be weighed against the importance of the evidence and any time pressures you're working against. Filename Similar to the subject line, the filename of an attachment can yield clues concerning the content within an attachment, especially when you know what you're looking for and when consistent file naming conventions are followed. So a search for financial reports might contain "fin" or "report" or "annual", while a search for Joe User's patient data might contain "Joe" or "JUser" or "JoeUser". Attachment content Finally, searching attachment content, while technically very challenging, can provide all the evidence you require to come to an HR or internal compliance-related decision, to respond to an SEC or other regulatory compliance-driven inquiry, or to provide evidence concerning a legal investigation. Whether a Word document confirming a breach of policy, or an Excel document containing records of illegal transactions, or any number of additional types of evidence, this is typically where you'll find most of the evidence required concerning a particular investigation.

So this rounds out our discussion of the different parts of a message, and hopefully today's blog underscores what the former SEC Chairman Arthur Levitt once said: "If you have the slightest question about it, store it". Needless to say, being able to rapidly search through all the e-mail parts we have discussed in order to find e-mail in all of the silos we have covered is a bit of a challenge.

Starting tomorrow we're going to start pulling together the various pieces we have covered thus far in order to dive into Exchange and compliance from a "proactive" perspective, namely one of establishing e-mail controls to protect the Exchange data that's important to your company.

Homework
Your homework for today is to take a moment to read the E-mail Compliance to-do list that I wrote in April for SearchExchange.com. Tomorrow's blog will be related to what I covered in that checklist.

Have a great day!

David
Posted by David Sengupta What's in a message? (... continued)
11 JUL 2005 00:38 EDT (04:38, GMT)
Recap
Last blog we talked about how an e-mail can be broken down into four parts, namely: (i) header; (ii) subject; (iii) body, and (iv) attachment (if one exists). We spent some time diving into the header, and learned of the importance of understanding the difference between P1 and P2 headers from a compliance perspective.

E-mail subject lines
Today we're going to take a look at the subject line and what it can tell us about a message.

E-mail message part

Importance to compliance

Subject Line

The subject line can provide a variety of information of interest from a compliance perspective. Indicator of content First off, and most obviously, people often (sometimes) fill in the subject line with a summary of what the message contains. So if you're tasked with searching e-mail for evidence concerning a particular topic, then looking through subject lines for related keywords would be a logical starting point. Thinking back to our "example breaches of compliance" (Examples 1 – 7 on day 2 of this blog ) some sample subject-line searches that might prove fruitful in turning up e-mail evidence might include: Example 1 Subject: "Furniture" AND "Order" Subject: "Order" AND "Received" Example 2 Subject: "HR" AND "Complaint" Example 3 Subject: "Joe User" Example 4 Subject: "Finance" AND "Audit" Conversational threads Secondly, if you're trying to understand how an e-mail passed through a given organization, you can piece together some of the conversational thread by finding all e-mails with a given subject line, then paying attention to the time and date-stamps along with the standard prefixes inserted by Outlook for typical actions (note that these are different for each localized version of Outlook... see below for English and German), including, as an example: Reply -- "RE:" (English), "AW:" (German), etc.Forward -- "FW:" (English), "WG" (German), etc. If Outlook is being used for searches of e-mail, then right-clicking any message and selecting "Find All" provides a quick way of finding all messages sharing the same subject line. If you're performing a compliance-driven investigation that focuses in on one or more e-mail users, you'll want to get an idea of who the suspect user frequently communicates with, discusses things with, and forwards e-mails to. This knowledge can prove valuable in piecing together the events that lead to whatever breach you're investigating. Indicator of read status Another factor in piecing together e-mail evidence is whether or not a message was read. By default "read receipt notifications" are not turned on; however, it's quite common for a user to put a "read receipt" on an e-mail before they send it in order to confirm that it's been read by the recipient. This is especially true for sensitive topics, and therefore any messages flagged with read receipts may be of interest to compliance investigators. Subject line keywords to search for include: "Read:" and "Not Read:." Needless to say, there are cases where a "Not Read:" receipt can play an role as an indicator of whether someone was privy to key information or not. Other system messages Finally, there are numerous other standard subject line keywords that are typically operations-focused but also relevant to compliance investigations, namely: Out of Office AutoReply: Provides insight into times an employee took time off or otherwise left the office (assuming Out-of-Office notification was set, of course). Undeliverable, DELIVERY FAILURE, etc.: Provide indication that a specific e-mail did not reach a particular recipient. Delivered: Confirms that a particular target mailbox received a specific e-mail Typically trolling through reams of e-mail as part of a compliance investigation provides a snapshot of a particular user's life, and paying attention to the various system messages found in that person's e-mail just serve to round out the picture of the environment the user was working in and any issues they were having sending e-mail to specific recipients or domains.

So the bottom line of what I wanted to communicate today is that knowing what you're searching for is really key to a compliance-driven investigation, and the tips and tricks I have shared will hopefully help you understand the importance of subject-line searches in narrowing down the evidence for your next compliance-driven investigation -- whenever it occurs.

One final note: If you find yourself frequently using Outlook's built-in "Find" functionality to search the contents of a particular mailbox, you should know that in pre-Outlook 2002 clients, the "Find" option uses a character-based search that recourses through all the e-mail in a particular mailbox, while the "Advanced Find" option uses a full-text index search, if enabled. In Outlook 2002 and higher, both "Find" and "Advanced Find" take advantage of full-text indexes, where they exist.

Tomorrow we'll wrap up our talk about message parts, focusing on searching the message body and attachments.

Have a great day!

David
Posted by David Sengupta What's in a message?
08 JUL 2005 14:44 EDT (18:44, GMT)
A moment of silence
I wrote today's entry on the day of the tragic events in London, so regardless of when you read this blog, I do want to take a moment to reflect. As with most of you, I was shocked and saddened by the bombings and loss of life that happened and just wanted to offer my deepest sympathies to any of you who were affected in any way. I stayed in a London hotel this past March that was less than a mile from where two of the bombings occurred, so the sights and sounds of Tavistock Square and King's Cross are still fresh in my memory. My prayer is for God to draw families who lost loved ones near to himself and for healing for those whose lives will be forever changed as a result of this tragedy.

It is really hard to transition from something so deeply tragic to something so theoretical and technical as Exchange and compliance, but I will move on.

Recap
Yesterday we spent some time breaking Exchange data into four "silos" of storage, namely: 1) online data, 2) backup data, 3) offline data in the file system and 4) archive data. Regardless of what is driving your compliance-related investigation, providing a complete set of evidence in support of your investigative efforts requires that you address each of these silos methodically. Today we'll have a quick primer in the parts of a message that you need to be concerned with for compliance purposes, and we'll look at one of those parts in depth.

What's in a message?
When you have a close look at an e-mail in your Inbox, you'll quickly come to see that it typically consists of four main parts:

1. the message header
2. the subject line
3. the message body
4. the attachment (if one exists).

Beginning our survey of message parts as they relate to compliance, let's have a look at message headers and why they're important to compliance.

E-mail message part

Importance to compliance

Message Header

The message header provides two really critical perspectives into each e-mail stored within your Exchange environment. There are really two parts to a header, namely the P1 and P2 headers. The P1 Header The P1 header is used to route a message and uses the equivalent of what is sent via the following commands (if you've ever used Telnet to port 25 of an e-mail gateway in order to test connectivity, this will be familiar): MAIL FROM: jim@badbadguycompany.com RCPT TO: joeuser@yourcompany.com RCPT TO: sallyuser@yourcompany.com While a P1 header is not stored in human-readable format, it really contains all the recipients of a specific message, whether these are addressed on the To: line, the Cc: line or the Bcc: line of an e-mail. From a compliance perspective, the P1 header is key because it's the only place where the actual recipients of any given message are recorded. So if I send a message to a distribution list, my To: field may just contain: To: everyoneoutthere@searchexchangereaders.com But this gives me absolutely no way of knowing who actually received this message (i.e., who was on the distribution list at the time of sending). P1 headers are not stored by default, but using something like Exchange 2003 Envelope Journaling enables capturing and storing P1 envelope data with every message for future reference. The bottom line is P1 headers are essential if you're attempting to prove that someone actually sent or received a specific message that is being considered as evidence for a given investigation. The P2 Header The P2 header is what you see when you open a particular message within Outlook, for example. So to continue our example above, consider the following stream of SMTP verbs: MAIL FROM: jim@badbadguycompany.com RCPT TO: joeuser@yourcompany.com RCPT TO: sallyuser@yourcompany.com DATA From: david@mydomain.com To: joeuser@yourcompany.com In this case the P2 header will make things appear as though the message actually originated from david@mydomain.com and was sent to joeuser@yourcompany.com, but there will be nothing in the P2 header indicating that the real sender of the message was jim@badbadguycompany.com…and also no indication of the fact that the message was Bcced to sallyuser@yourcompany.com. So if you're in the midst of collecting evidence and you need to go to court to prove who actually sent the message to joeuser@yourcompnay.com, you really need to use the "correct" information from the P1 header, instead of the possibly spoofed information from the P2 header.

As we progress through the remaining six days of this Expert Answer Center blog on Exchange compliance, we'll be looking at archival and touching on journaling, and this background concerning P1 and P2 headers will be important. Stay tuned.

Tomorrow we'll continue our survey of message parts to take a look at the content in a message, along with some tips on what kinds of information are important to compliance.

There will be no Quick Poll today.

David
Posted by David Sengupta Data, data everywhere …
07 JUL 2005 15:53 EDT (19:53, GMT)
Recap
Yesterday we spent some time looking at example breaches of compliance, for each of the three compliance regimes:

1. compliance with corporate policies
2. regulatory compliance and
3. legal compliance

Data, data everywhere …
When you think of managing the Exchange data in your environment, there are really four main "silos" that you need to keep in mind. Your ability to respond effectively to a compliance-related inquiry or investigation really comes down to how effective you are managing the Exchange data in each of these silos. So much of "compliance" really boils down to having effective storage management policies, procedures and technologies in place.

Here are the four e-mail storage "silos":

E-mail storage "silo"	Description
Online data	E-mail stored in product Exchange servers, namely within mailboxes or public folders.
Backup data	E-mail stored in backups on backup media -- tapes, snapshots, disk-based backup, other media -- whether daily, weekly, monthly or yearly backups and be they full, incremental or differential backups.
Offline data in the file system	E-mail stored in the file system, within PST files, OST files, MSG files or otherwise saved to disk. This also includes mobile devices, be they Blackberry, iPAQ, SmartPhone or other devices.
Archive data	E-mail stored in an archival system for either compliance- or storage-management purposes.

So as you think through the various examples of compliance breaches we discussed yesterday, you'll soon come to realize that responding effectively to provide evidence concerning each breach requires effective search across each of these silos. On the other hand, the careful reader will also note that retaining data in each of these silos without any checks and balances can put your company at risk; saving only what you need to save is typically a good practice.

Quick Poll #3
For today's Quick Poll please create a message to me, then copy the questions below into the message, answer the questions and send it to me (your name, company and other personal information will not be published).

Quick Poll #3 Questions:

1. Rank the silos above in order of importance to your company from a compliance perspective (1 is most important; 4 is least important).
2. How much PST data exists in your organization 1) on file servers and (2) on laptops and PCs?

David
Posted by David Sengupta Am I affected?
06 JUL 2005 08:14 EDT (12:14, GMT)
Recap
Yesterday we spent some time defining compliance and came to the conclusion that compliance can be broken down into compliance with corporate policies, regulatory compliance and legal compliance. Today we'll have a look at ways in which each of these compliance "regimes" can impact your Exchange environment.

Am I affected?
When defined as above, compliance affects any company regardless of size, geographic location and whether you're publicly traded, privately held or a governmental organization. Every company in existence falls under the realm of one of these types of compliance. Let's take each compliance "regime" and consider some example breaches of various compliance regimes as they could well happen (or may be happening today!) within your corporate Exchange environment. I won't get into the specifics of each violation, though I will indicate which policy, regulation or type of law is being breached in parentheses, in each case. (These examples are completely fictitious.)

Compliance regime	Example of breach of compliance
Compliance with corporate policies	Example 1. Joe User is running a furniture business on the side and decides to use the corporate Microsoft Exchange infrastructure to accept orders and to deal with his suppliers. (email usage policy) Example 2. Sally User sends an e-mail over the corporate Microsoft Exchange infrastructure to her manager using racially discriminatory language and derogatory inferences in referring to Joe User. (HR policy)
Regulatory compliance	Example 3. Dr. Smith uses the corporate Exchange infrastructure to send a detailed diagnosis of Joe User's medical history to Sally User for advice. Dr. Smith has not obtained consent from Joe User. Dr. Smith is also not using encrypted e-mail to send this information to Sally User over the Internet. (Health Insurance Portability and Accountability Act of 1996 [HIPAA]) Example 4. Joe User has a PST file containing internal discussions amongst financial auditors about financial audits for his employer, an Australian branch office of a company that is publicly traded on a U.S. stock exchange. Joe deletes the PST in order to free up space on his local hard drive. (Sarbanes-Oxley Act of 2002)
Legal compliance	Example 5. Sally User sends a message to a list of prospective customers advertising a new product from her company. Sally User uses an Outlook Express client to relay off her Exchange server, spoofing the "from" address and setting the "reply-to" address to a disguised free Internet e-mail address in order to try to increase product sales for the quarter. (Florida Electronic Mail Communications Act of 2004) Example 6. Jim Badbadguy works in your company and uses your corporate Exchange infrastructure to discuss plans with some of his Internet pals to destroy critical infrastructure in your local municipality. (For the scope of this discussion, critical infrastructure includes energy and utilities [electrical power, natural gas, oil transmission systems], communications systems [telecommunications and broadcasting], financial services, food distribution and health care systems, transportation systems [air, rail, marine and surface transportation], nuclear power facilities, search and rescue facilities, emergency services, government facilities and government networks and other assets.) (Canadian Criminal Code) Example 7. Joe User resigned his position over a dispute with Sally User. Seven months following his resignation, Joe files a lawsuit against your company, and as part of the court proceedings, your company has been given 90 days to produce all evidence associated with this case. This includes two specific e-mail threads between Joe User and Sally User spanning approximately 80 days that occurred about four years ago that have been subpoenaed specifically. (Court order)

As you've read through the examples above, I hope that you've come to see that compliance is not simply a buzzword driven by recent regulatory hype, but affects every corporate e-mail environment, whether this is formally acknowledged within your company or not. Whether explicitly defined or simply assumed, your company is responsible for its Exchange infrastructure; how it is used, who has access to it and the data it contains. Whether simply having to observe the laws and statutes that exist in your jurisdiction, whether having to apply internal corporate policies governing mailbox quotas or whether having to retain every e-mail that passes through your company globally for seven years, it is a best practice to be proactive about Exchange and compliance to ensure you are ready to comply when -- not if -- a compliance-related investigation comes your way. Tomorrow we will have a look at the scope of corporate Exchange data that you need to take into consideration from a compliance perspective.

Quick Poll #2
Our Quick Poll for day is a simple one, and once again one that will help me to ensure my articles align with your interests. Create a message to me, then copy the questions below into the message, answer the questions and send it to me (your name, company and other personal information will not be published).

Quick Poll #2 Questions:

1. Take the seven examples I've listed above and list them in order of interest to you and your company (where 1 is most important and 7 is least important).
2. Have you ever encountered a compliance breach similar to any of the examples I listed? [yes/no]
3. If so, which one(s)?

David
Posted by David Sengupta Welcome to the Microsoft Exchange compliance blog!
04 JUL 2005 19:38 EDT (23:38, GMT)
Welcome to the Expert Answer Center's Exchange compliance topic! Our focus for the following two weeks will be on e-mail compliance. Much of what we cover will be messaging system agnostic, but I'll pull out the Exchange-specific implications where it makes sense.

If you've followed the news much or had dealings with any of the major archive-, storage- or application-management vendors, you'll probably have seen frequent mention of "compliance." Typically compliance discussions center around financial systems and associated processes, the need to establish business controls governing access to sensitive financial data, and the establishment of audit trails to ensure traceability in the event of a breach of policy or security. With high-profile cases such as Enron and Morgan Stanley getting a lot of news media coverage and with e-mail usage increasing virtually exponentially, more and more companies are finding themselves faced with having to produce e-mail as evidence for any number of reasons.

In my role as an Exchange Most Valuable Professional (MVP) and as an Exchange site expert for SearchExchange.com, I spend a significant amount of time reviewing the many questions facing Exchange administrators and IT managers around the globe, and I get a good sense of the high-frequency issues out there and the kinds of challenges companies are grappling with day in and day out. I also have the privilege of interfacing regularly with many of the other Exchange experts in the industry and with various Exchange communities within Microsoft to hear of the Exchange challenges they are facing "in the wild." Finally, in my role as a product manager for a major Exchange management solutions vendor (Quest Software), I spend much of my days listening to customers articulate Exchange management priorities and challenges they are facing -- often in some of the largest Exchange organizations in the world.

Over the past 18 months, "compliance" has surfaced as an issue in many discussions I have been a part of across North America, Europe and the Pacific Rim. I hope to use these two weeks to share some of what I have learned and a bit of where the industry as a whole has been going in the compliance arena, specific to e-mail. Most importantly, experience has shown that while much of the compliance arena has done a great job of addressing compliance, specifically to financial systems and accounting practices, e-mail systems and the data they contain are largely ignored or overlooked in the controls companies are establishing. As such, much of what we'll cover is intended as thought leadership. I am not a lawyer, and as such you need to vet what we'll cover against your corporate legal counsel where appropriate. But hopefully these two weeks will help you understand the issues at stake…and hopefully we will have some good discussions around the implications on Exchange infrastructures as a whole.

Before we start into the issues, I just wanted to level with you and to set everyone's expectations to explain the scope of what I'll be addressing in this Expert Answer Center topic. Today we'll look at what is meant by "compliance."

What is meant by "compliance?"
According to Merriam-Webster's Dictionary of Law (©1996 Merriam-Webster Inc.), "compliance" is defined as "an act or process of complying with a demand or recommendation."

From an information technology perspective, there are three common sources for these kinds of "demands or recommendations," namely internal corporate policies, external regulatory bodies and laws or statutes. In other words, compliance can be broken down into

• compliance with corporate policies
• regulatory compliance and
• legal compliance.

Quick Poll #1
As part of our theme this week, we'll be conducting some "Quick Polls" in order to get some audience feedback going and to help me ensure that I'm touching on the topics of interest to you. So your optional "homework" for today is to create a message to me, then copy the questions below into the message, answer the questions and send it to me (your name, company and other personal information will not be published).

Quick Poll #1 Questions:

1. Rank the following compliance regimes in order of importance [1 is most important; 2 is second in importance; 3 is least important).
   • Compliance with corporate policies
   • Regulatory compliance
   • Legal compliance
2. Have you ever had to investigate someone's e-mail as part of a compliance investigation of some sort? [yes/no]

David
Posted by David Sengupta