Category Archives: Exchange 2010

Slow Migration – Office 365

The story:

In office 365 when you’re working on Exchange 2010,2013, 2016 or 2019 in a hybrid environment things might look easy but in a big enterprises where Internet security is something being taken into account very seriously. It might cause many issues that you don’t expect at all.

One of my clients whom I was doing Exchange Migration for had an issue with the Migration. The error was as follows:

Error occurs after Office 365 Exchange online connects to Exchange on-premises 2010 mailbox server

Error in Office 365

         : 20.

                                           27.04.2016 08:03:17 [DB3PR05MB0778] Transient error DataExportTransientExcep

                                           tion has occurred. The system will retry (2/1280).

                                           27.04.2016 08:04:53 [DB3PR05MB0778] The Microsoft Exchange Mailbox Replicati

                                           on service ‘DB3PR05MB0778.eurprd05.prod.outlook.com’ (15.1.466.25 caps:03FFF

                                           F) is examining the request.

                                           27.04.2016 08:04:55 [DB3PR05MB0778] Connected to target mailbox ‘lcwonline.o

                                           nmicrosoft.com\ec96e315-1059-4710-b358-1c4b42f3edeb (Primary)’, database ‘EU

                                           RPR05DG049-db131′, Mailbox server ‘DB3PR05MB0778.eurprd05.prod.outlook.com’

                                           Version 15.1 (Build 466.0).RequestExpiryTimestamp                   : 03.04.2116 07:42:38

ObjectState                              : New

Troubleshooting:

To troubleshoot issues, You need to put so many things into account! The architecture of the infrastructure of where you are doing the project is very important and the need of knowing how things are working matters.

Things that could always come in mind and handy are what you will need to start your troubleshooting:

– Bandwidth Limitations or Performance:

https://technet.microsoft.com/en-us/library/dn592150(v=exchg.150).aspx

https://technet.microsoft.com/en-us/library/jj200581(v=exchg.150).aspx

– Exchange Configuration (MRS)

To troubleshoot the MRs, You need to know what kind of error you’re getting and to see this you can use the following powershell after you connect to Office 365 powershell.

Get-MoveRequest {email} | Get-MoveRequestStatistics -Diagnostic -IncludeReport | Export-Clixml c:\logfile.xml

The resultant report will reveal the error and shows you where is the exact culprit.

– Disk Latency
– Firewall Configuration (IPS/IDS)

From Exchange 2016 to 2019 or 2013 to 2016 The transient error might be related to MRSProxy or at least this is the case with me 90% of the time. To resolve this issue you will need to change the MRSProxy values on the target server and depending on the error might also be the Source server too.

SOLUTION:

===========

1. Some instability was detected in communications as well as saturation by the size of the link.
2. The procedure to increase the timeout for the service through the file MRSProxy

File: MsExchangeMailboxReplication.exe.config

Object / line: DataImportTimeout.

New Value: 00:10:00

clip_image001[4]

New Configuration

clip_image001[6]

Offline Address book Issue after migration from Exchange 2003 to Exchange 2010

Error: event ID: 9360

OABGen encountered an error while generating the changes.oab file for version 2 and 3 differential downloads of address list

RESOLUTION 1

Important This section, method, or task contains steps that tell you how to modify the registry. However, serious problems might occur if you modify the registry incorrectly. Therefore, make sure that you follow these steps carefully. For added protection, back up the registry before you modify it. Then, you can restore the registry if a problem occurs. For more information about how to back up and restore the registry, click the following article number to view the article in the Microsoft Knowledge Base:

322756 How to back up and restore the registry in Windows

To resolve this issue, follow these steps on the server that is running Exchange Server 2003:

  1. Click Start, click Run, type regedit in the Open box, and then click OK.
  2. Locate and then right-click the following registry subkey:
  3. HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeSA\Parameters
  4. Point to New, and then click DWORD Value.
  5. Type OAL post full if diff fails to name the new value.
  6. Right-click OAL post full if diff fails, and then click Modify.
  7. In the Value data box, type 1, and then click OK.
  8. Exit Registry Editor.
  9. Dismount and then mount the Public Folder Store again. To dismount and then mount the public folder store, follow these steps:
    1. Start Exchange System Manager.
    2. Expand Servers, expand the server that you want, expand Storage_Group_Name, and then right-click Public Folder Store.
    3. Note If administrative groups are defined, follow these steps:
      • Expand Administrative Groups.
      • Expand Administrative_Group_Name.
      • Expand Servers.
      • Expand the server that you want.
      • Expand Storage_Group_Name.
      • Right-click Public Folder Store.
    4. Click Dismount Store, and then click Yes to continue.
    5. Right-click Public Folder Store, click Mount Store, and then click OK.

A new parent Legacy Exchange DN container value ‘/o=HEMA/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients’ was found during generation of the differential update file for offline address list ‘\Global Address List’. This will force clients using this offline address list to do a full download of the offline address list.

– \Default Offline Address List

Resolution 2:

If the first resolution didn’t work, try to disable version 2 and 3 OAB, then update again and see if you get any errors.

Note that this may take some time to take effect.

Exchange 2010 to 2013 Migration fails with “You cannot have ArchiveDomain set when archive is not enabled for this user”

I have previously done a Hybrid integration with Office 365 with my Exchange 2010 server and enabled Archiving online when I migrated my user to Exchange online but then I finished my demo and decided to bring the user back on-premises. Now I have deployed Exchange 2013 and wanted to migrate the same user to Exchange 2013 from 2010 but the migration request fails with the following message.

clip_image001

6/7/2015 1:23:24 PM [EXCH2K13] ” created move request.

6/7/2015 1:23:57 PM [EXCH2K13] The Microsoft Exchange Mailbox Replication service ‘EXCH2K13.demotesas.local’ (15.0.1076.6 caps:1FFF) is examining the request.

6/7/2015 1:23:59 PM [EXCH2K13] Connected to target mailbox ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’, database ‘Mailbox Database 0439787427’, Mailbox server ‘EXCH2K13.demotesas.local’ Version 15.0 (Build 1076.0).

6/7/2015 1:23:59 PM [EXCH2K13] Connected to source mailbox ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’, database ‘Database1’, Mailbox server ‘EXCH01.demotesas.local’ Version 14.3 (Build 174.0).

6/7/2015 1:23:59 PM [EXCH2K13] Request processing started.

6/7/2015 1:23:59 PM [EXCH2K13] Source mailbox information:

Regular Items: 104, 5.549 MB (5,818,789 bytes)

Regular Deleted Items: 0, 0 B (0 bytes)

FAI Items: 50, 0 B (0 bytes)

FAI Deleted Items: 0, 0 B (0 bytes)

6/7/2015 1:23:59 PM [EXCH2K13] Cleared sync state for request b6ee5dd7-beab-45a0-9933-8e926a694de3 due to ‘CleanupOrphanedMailbox’.

6/7/2015 1:23:59 PM [EXCH2K13] Mailbox signature will not be preserved for mailbox ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’. Outlook clients will need to restart to access the moved mailbox.

6/7/2015 1:24:04 PM [EXCH2K13] Stage: CreatingFolderHierarchy. Percent complete: 10.

6/7/2015 1:24:05 PM [EXCH2K13] Initializing folder hierarchy from mailbox ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’: 76 folders total.

6/7/2015 1:24:05 PM [EXCH2K13] Folder creation progress: 0 folders created in mailbox ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’.

6/7/2015 1:24:10 PM [EXCH2K13] Folder hierarchy initialized for mailbox ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’: 75 folders created.

6/7/2015 1:24:10 PM [EXCH2K13] Stage: CreatingInitialSyncCheckpoint. Percent complete: 15.

6/7/2015 1:24:10 PM [EXCH2K13] Initial sync checkpoint progress: 0/76 folders processed. Currently processing mailbox ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’.

6/7/2015 1:24:12 PM [EXCH2K13] Initial sync checkpoint completed: 66 folders processed.

6/7/2015 1:24:12 PM [EXCH2K13] Stage: LoadingMessages. Percent complete: 20.

6/7/2015 1:24:14 PM [EXCH2K13] Messages have been enumerated successfully. 154 items loaded. Total size: 5.55 MB (5,819,724 bytes).

6/7/2015 1:24:14 PM [EXCH2K13] Stage: CopyingMessages. Percent complete: 25.

6/7/2015 1:24:14 PM [EXCH2K13] Copy progress: 0/154 messages, 0 B (0 bytes)/5.55 MB (5,819,724 bytes), 55/76 folders completed.

6/7/2015 1:24:58 PM [EXCH2K13] Copying messages is complete. Copying rules and security descriptors.

6/7/2015 1:25:04 PM [EXCH2K13] Initial seeding completed, 154 items copied, total size 5.55 MB (5,819,724 bytes).

6/7/2015 1:25:04 PM [EXCH2K13] Stage: IncrementalSync. Percent complete: 95.

6/7/2015 1:25:05 PM [EXCH2K13] Folder hierarchy changes reported in source ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’: 2 changed folders, 0 deleted folders.

6/7/2015 1:25:05 PM [EXCH2K13] Content changes reported for mailbox ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’: Batch 1, New 3, Changed 1, Deleted 0, Read 0, Unread 0, Total 4.

6/7/2015 1:25:05 PM [EXCH2K13] Total content changes applied to mailbox ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’: New 3, Changed 1, Deleted 0, Read 0, Unread 0, Skipped 0, Total 4.

6/7/2015 1:25:05 PM [EXCH2K13] Incremental Sync ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’ completed: 2 hierarchy updates, 4 content changes.

6/7/2015 1:25:05 PM [EXCH2K13] Stage: IncrementalSync. Percent complete: 95.

6/7/2015 1:25:07 PM [EXCH2K13] Final sync has started.

6/7/2015 1:25:07 PM [EXCH2K13] Folder hierarchy changes reported in source ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’: 0 changed folders, 1 deleted folders.

6/7/2015 1:25:07 PM [EXCH2K13] Incremental Sync ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’ completed: 1 hierarchy updates, 0 content changes.

6/7/2015 1:25:07 PM [EXCH2K13] Source mailbox information:

Regular Items: 108, 5.562 MB (5,832,087 bytes)

Regular Deleted Items: 0, 0 B (0 bytes)

FAI Items: 50, 0 B (0 bytes)

FAI Deleted Items: 0, 0 B (0 bytes)

6/7/2015 1:25:07 PM [EXCH2K13] Stage: FinalIncrementalSync. Percent complete: 95.

6/7/2015 1:25:09 PM [EXCH2K13] Mailbox store finalization is complete.

6/7/2015 1:25:09 PM [EXCH2K13] SessionStatistics updated.

6/7/2015 1:25:09 PM [EXCH2K13] Verifying mailbox contents…

6/7/2015 1:25:10 PM [EXCH2K13] Mailbox contents verification complete: 66 folders, 157 items, 5.562 MB (5,831,953 bytes).

6/7/2015 1:25:10 PM [EXCH2K13] Mailbox ‘Mohammed JA. Hamada’ was loaded from domain controller ‘ad.demotesas.local’.

6/7/2015 1:25:18 PM [EXCH2K13] Fatal error UpdateMovedMailboxPermanentException has occurred.

On Exchange 2010, I launched Exchange Management shell and ran the following cmdlet which will show any attribute that has arch in it for the user Mohammed

Get-mailbox User | fl arch*

clip_image002

Since there’s no archive mailbox then the archive domain is invalid and I don’t even own it anymore as it has expired a while ago.

Resolution:

I will try to remove the archive domain object from the user’s properties using the following cmdlet

Set-mailbox mailboxname -ArchiveDomain $null

clip_image003

Using the above cmdlet seems to fail due to this property being administered by Exchange server so it’ll have to be removed manually.

I will open the user’s attribute and delete the value and try to continue the migration again.

clip_image004

I’ll click on Edit then Clear and OK

clip_image005
clip_image006
clip_image007

Migration finished successfully

clip_image008
clip_image009

Hope this helps

Exchange 2010 Performance counters for the Client access role is not installed

Performance counters for the Client access role is not installed

To solve the problem

  • Open the Exchange Management Shell
  • Run the following cmd: add-pssnapin Microsoft.Exchange.Management.PowerShell.Setup
  • Run the following cmd: new-perfcounters –definitionfilename “C:\Program Files\Microsoft\Exchange Server\V14\Setup\Perf\RpcClientAccessPerformanceCounters.xml”

By running these cmds we will install the Performance Counters needed for the RPC Client Access Service. Once installed the error won’t be displayed anymore.

Prepare Schema for Exchange 2013 Migration while having Hybrid Integration with Exchange 2010

In a very interesting situation that I came through I had an environment with two DCs and Exchange 2010 that I had previously setup for Hybrid integration with Office 365 for demonstration with a trial subscription but I haven’t removed the integration after I finished my test and the trial expired and the tenant was deleted.

Next I intended to upgrade my existing Exchange 2010 to Exchange 2013 and setup coexistence between them however, I have stumbled in the step of preparation of AD schema for Exchange 2013. While trying to prepare the schema I got the following error

clip_image001

Setup /PrepareSchema /IAcceptExchangeServerLicenseTerms

Welcome to Microsoft Exchange Server 2013 Cumulative Update 8 Unattended Setup

Copying Files…

File copy complete. Setup will now collect additional information needed for

installation.

Performing Microsoft Exchange Server Prerequisite Check

Prerequisite Analysis FAILED

A hybrid deployment with Office 365 has been detected. Please ensure that you are running setup with the /TenantOrganizationConfig switch. To use the TenantOrganizationConfig switch you must first connect to your Exchange Online tenant via PowerShell and execute the following command: “Get-OrganizationConfig | Export-Clixml -Path MyTenantOrganizationConfig.XML”. Once the XML file has been generated, run setup with the TenantOrganizationConfig switch as follows “/TenantOrganizationConfig MyTenantOrganizationConfig.XML”.

If you continue to see this this message then it indicates that either the XML file specified is corrupt, or you are attempting to upgrade your on-premises Exchange installation to a build that isn’t compatible with the Exchange version of your Office 365 tenant. Your Office 365 tenant must be upgraded to a compatible version of Exchange before upgrading your on-premises Exchange installation. For

more information, see: http://go.microsoft.com/fwlink/?LinkId=262888

For more information, visit: http://technet.microsoft.com/library(EXCHG.150

)/ms.exch.setupreadiness.DidTenantSettingCreatedAnException.aspx The Exchange Server setup operation didn’t complete. More details can be found in ExchangeSetup.log located in the <SystemDrive>:\ExchangeSetupLogs folder.

The Office 365 Hybrid setup was still there in my Exchange Console and since I couldn’t follow MS’s recommended steps to connect to O365 tenant and get the XML file then I had to do things manually.

First I connected to the EMC and removed all the instances that were created during the Exchange Hybrid Wizard Configuration

1- Removing Organization Relationships

clip_image001[4]
image

2- Removing Federation Trust

clip_image001[6]
clip_image002

3- Removing Remote Domains

clip_image001[8]
clip_image002[4]

4- Removing Accepted Domains

clip_image001[10]
clip_image002[6]

5- Removing Send and Receive Connectors

clip_image001[12]
clip_image002[8]
clip_image003

6- Lastly the Hybrid Configuration object…

Since remove-hybridconfiguration cmdlet is not supported to remove the hybrid configuration object from AD then we have no choice but to use ADSIEDIT tool to do so.

I will navigate to Configuration > Services > Microsoft Exchange > First Organization > Delete “CN=Hybrid Configuration”

image
image
image

Restart MSExchangeServicehost

clip_image001[14]
image

Now I will try again to prepare AD schema for Exchange 2013 but I got a different error

clip_image001[16]

Extending Active Directory schema FAILED

The following error was generated when “$error.Clear);

install-ExchangeSchema -LdapFileName ($roleInstallPath + “Setup\Data\”+$

RoleSchemaPrefix + “schema0.ldf”)” was run: “Microsoft.Exchange.Configuration.Tasks.TaskException: There was an error while running ‘ldifde.exe’ to import the schema file ‘C:\Windows\Temp\ExchangeSetup\Setup\Data\PostExchange2003_schema0.ldf’. The error code is: 8224. More details can be found in the error file: ‘C:\Users\Administrator.DEMOTESAS\AppData\Local\Temp\2\ldif.err’at Microsoft.Exchange.Configuration.Tasks.Task.ThrowError(Exception exception, ErrorCategory errorCategory, Object target, String helpUrl)at Microsoft.Exchange.Management.Deployment.InstallExchangeSchema.ImportSchem

aFile(String schemaMasterServer, String schemaFilePath, String macroName, StringmacroValue, WriteVerboseDelegate writeVerbose)at Microsoft.Exchange.Management.Deployment.InstallExchangeSchema.InternalPro

cessRecord()at Microsoft.Exchange.Configuration.Tasks.Task.<ProcessRecord>b__b()at Microsoft.Exchange.Configuration.Tasks.Task.InvokeRetryableFunc(String funcName, Action func, Boolean terminatePipelineIfFailed)”.The Exchange Server setup operation didn’t complete. More details can be found

in ExchangeSetup.log located in the <SystemDrive>:\ExchangeSetupLogs folder.

Checking the ldif.err file mentioned in the error above it seems that Exchange is complaining because the changes of the AD schema is not being replicated to the other AD partners which is true since I have another additional DC that’s turned off.

clip_image001[19]

After turning on the other DC we’ll see what happens

The other DC had another issue as I have turned it off for long time and it was not syncing due to expired Tomb stone life so I had to fix this issue as well and I have published it in a different article.

Please click here to see how the replication issue was fixed.

Issue has been fixed

clip_image001[21]
clip_image002[10]

Hope someone finds this useful

Exchange 2010 Console after DC migration stopped working

After DC migration and changing in the PDC and Schema master role server to the new DC and shut down the old DC for test. On Exchange 2010 server you might get the following error

An error caused a change in the current set of Active Directory settings. Restart The Exchange Management Console.

Exchange Console

clip_image019

Current deployment

  1. Exchange 2010
  2. New DC 2012 R2 with another Additional DC installed newly.
  3. Two DC 2008R2 but have been shut down for testing.

Problem:

After you shutdown or demote the old PDC or Schema master Demote Domain Controller role, Microsoft Exchange Management Console fails to retrieve any Exchange information with error message “An error caused a change in the current set of Active Directory Server settings. Restart Exchange Management console.”

Cause

Microsoft Exchange management console caches the data in the user’s profile for quick access, So whenever you try to open EMC from an existing Exchange admin profile you will get the same error.

Resolution:

Navigate to the following folder and delete the Exchange Management Console file.

%userprofile%\appdata\roaming\Microsoft\MMC\Exchange Management Console

clip_image020

Close EMC and reopen it and you should be done.

Exchange 2007/2010 Doesn’t show new DC (2012) servers after adding them as additional DCs

Symptoms

In an environment where one DC exist after adding Windows 2012 R2 Servers as additional servers, Exchange 2007 doesn’t show the new servers although they also hold GC.

image

Research:

To locate the problem you should search the event ID (2080) which shows the populated DCs and the permissions allowed on Exchange servers

In the below screenshot, the SACL right was not provided to the new DCs due to GPO problem.

image

After checking sites, Replication, all is healthy and no issue with it.

3 servers (Two 2012 servers) and one DC 2003 Server

Exchange 2010 SP3 servers.

image

Reason:

The Default Domain Controllers Policy was not linked to the Domain Controllers OU.

image
image
image

Resolution:

After Linking the Domain Controllers OU to the Default Controllers policy, the SACL permission was provided without any issue.

image

Now Exchange is reporting healthy and can read the new DCs which allow us to demote the old DCs

image

After removing the old DC

image