Category Archives: Microsoft

Messaging and Collaboration, Sharepoint.. etc

Exchange 2013 OWA,Async,And OA error MsExchange BackEndRehydration event id 3002

Users can’t access their mailboxes from anywhere as they get the error in the screenshot.

Related errors are 3002, 3005

Event code: 3005

Event message: An unhandled exception has occurred.

Event time: 7/29/2015 11:10:57 AM

Event time (UTC): 7/29/2015 8:10:57 AM

Event ID: 6f94ea40e3964fb1a05d9fc48ffb4299

Event sequence: 38

Event occurrence: 2

Event detail code: 0

Application information:

Application domain: /LM/W3SVC/1/ROOT/owa-2-130826309519814020

Trust level: Full

Application Virtual Path: /owa

Application Path: C:\Program Files\Microsoft\Exchange Server\V15\FrontEnd\HttpProxy\owa\

Machine name: EX2K1301

Process information:

Process ID: 19348

Process name: w3wp.exe

Account name: NT AUTHORITY\SYSTEM

Exception information:

Exception type: NullReferenceException

Exception message: Object reference not set to an instance of an object.

at Microsoft.Exchange.Clients.Common.UserAgent.HasString(String str)

at Microsoft.Exchange.Clients.Common.UserAgent.get_Layout()

at Microsoft.Exchange.Clients.Common.UserAgent.get_LayoutString()

at ASP.auth_logon_aspx.__Render__control1(HtmlTextWriter __w, Control parameterContainer)

at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)

at System.Web.UI.Page.Render(HtmlTextWriter writer)

at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)

at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

Request information:

Request URL: https://mail.Domain.com:443/owa/auth/logon.aspx?url=https://mail.Domain.com/owa/PowerShell-LiveID&reason=0

Request path: /owa/auth/logon.aspx

User host address: 10.16.0.172

User:

Is authenticated: False

Authentication Type:

Thread account name: NT AUTHORITY\SYSTEM

Thread information:

Thread ID: 67

Thread account name: NT AUTHORITY\SYSTEM

Is impersonating: False

Stack trace: at Microsoft.Exchange.Clients.Common.UserAgent.HasString(String str)

at Microsoft.Exchange.Clients.Common.UserAgent.get_Layout()

at Microsoft.Exchange.Clients.Common.UserAgent.get_LayoutString()

at ASP.auth_logon_aspx.__Render__control1(HtmlTextWriter __w, Control parameterContainer)

at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)

at System.Web.UI.Page.Render(HtmlTextWriter writer)

at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)

at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

Custom event details:

OWA PAGE

Solution:

On Exchange servers, Make sure that Exchange servers are not members of Organization Management group and if they are then remove them and run this cmdlet anyway on all Exchange Servers then restart the Servers.

Get-ClientAccessServer | Add-ADPermission -AccessRights ExtendedRight -ExtendedRights “ms-Exch-EPI-Token-Serialization”, “ms-Exch-EPI-Impersonation” -User (Exchange Server name)

Make sure you restart Exchange servers after you apply these cmdlet

Exchange 2013 ECP fails with 500 Unexpected error after running Hybrid Configuration Wizard with Office 365

Symptoms

After you attempt to try Hybrid Configuration Wizard between Exchange 2013 SP1 and Exchange online (Office 365), You are unable to login to your OWA/ECP Page and instead you get an 500 unexpected error:

clip_image001

If you go to event viewer You might find Event ID 4 which shows the error:

clip_image002

Cause:

The HCW or “Hybrid Configuration Wizard” In Exchange 2013 (CU6 or 8) might cause some changes to your CAS folder in the following path and file.

“c:\Program Files\Microsoft\Exchange Server\V15\ClientAccess\ecp\DDI”` the file name is RemoteDomains.xaml and multiply some variables which causes ECP to fail and report that error.

Error:


Current user: 'Domain.local/User'

Request for URL 'https://ex2k1301.Domain.local:444/ecp/default.aspx(https://mail.Domain.com/ecp/)' failed with the following error:

System.Web.HttpUnhandledException (0x80004005): Exception of type 'System.Web.HttpUnhandledException' was thrown. ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Data.DuplicateNameException: A column named 'TargetDeliveryDomain' already belongs to this DataTable.

at System.Data.DataColumnCollection.RegisterColumnName(String name, DataColumn column)

at System.Data.DataColumnCollection.BaseAdd(DataColumn column)

at System.Data.DataColumnCollection.AddAt(Int32 index, DataColumn column)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase.CreateColumn(DataTable table, Dictionary`2 rbacMetaData)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase..ctor(Service profileBuilder)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase..ctor(String schemaFilesInstallPath, String schema)

at Microsoft.Exchange.Management.DDIService.WSListDataHandler..ctor(String schemaFilesInstallPath, String resourceName, String workflowName, DDIParameters parameters, SortOptions sortOptions)

at Microsoft.Exchange.Management.DDIService.DDIServiceHelper.GetListCommon(DDIParameters filter, SortOptions sort, Boolean forGetProgress)

--- End of inner exception stack trace ---

at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)

at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)

at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

at Microsoft.Exchange.Management.ControlPanel.WebServiceReference.GetList(DDIParameters filter, SortOptions sort)

at Microsoft.Exchange.Management.ControlPanel.OrganizationCache.LoadTargetDeliveryDomain(AddValueHandler addValue, LogErrorHandler logError)

at Microsoft.Exchange.Management.ControlPanel.OrganizationCache.TryGetValue[T](String key, T& value)

at Microsoft.Exchange.Management.ControlPanel._Default.RenderMetroTopNav()

at ASP.default_aspx.__RendermainForm(HtmlTextWriter __w, Control parameterContainer)

at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)

at System.Web.UI.HtmlControls.HtmlForm.RenderChildren(HtmlTextWriter writer)

at System.Web.UI.HtmlControls.HtmlContainerControl.Render(HtmlTextWriter writer)

at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)

at ASP.default_aspx.__Render__control1(HtmlTextWriter __w, Control parameterContainer)

at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)

at System.Web.UI.Page.Render(HtmlTextWriter writer)

at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)

at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

at System.Web.UI.Page.HandleError(Exception e)

at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

at System.Web.UI.Page.ProcessRequest()

at System.Web.UI.Page.ProcessRequest(HttpContext context)

at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()

at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

at System.Web.UI.Page.HandleError(Exception e)

at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

at System.Web.UI.Page.ProcessRequest()

at System.Web.UI.Page.ProcessRequest(HttpContext context)

at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()

at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Data.DuplicateNameException: A column named 'TargetDeliveryDomain' already belongs to this DataTable.

at System.Data.DataColumnCollection.RegisterColumnName(String name, DataColumn column)

at System.Data.DataColumnCollection.BaseAdd(DataColumn column)

at System.Data.DataColumnCollection.AddAt(Int32 index, DataColumn column)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase.CreateColumn(DataTable table, Dictionary`2 rbacMetaData)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase..ctor(Service profileBuilder)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase..ctor(String schemaFilesInstallPath, String schema)

at Microsoft.Exchange.Management.DDIService.WSListDataHandler..ctor(String schemaFilesInstallPath, String resourceName, String workflowName, DDIParameters parameters, SortOptions sortOptions)

at Microsoft.Exchange.Management.DDIService.DDIServiceHelper.GetListCommon(DDIParameters filter, SortOptions sort, Boolean forGetProgress)

--- End of inner exception stack trace ---

at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)

at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)

at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

at Microsoft.Exchange.Management.ControlPanel.WebServiceReference.GetList(DDIParameters filter, SortOptions sort)

at Microsoft.Exchange.Management.ControlPanel.OrganizationCache.LoadTargetDeliveryDomain(AddValueHandler addValue, LogErrorHandler logError)

at Microsoft.Exchange.Management.ControlPanel.OrganizationCache.TryGetValue[T](String key, T& value)

at Microsoft.Exchange.Management.ControlPanel._Default.RenderMetroTopNav()

at ASP.default_aspx.__RendermainForm(HtmlTextWriter __w, Control parameterContainer)

at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)

at System.Web.UI.HtmlControls.HtmlForm.RenderChildren(HtmlTextWriter writer)

at System.Web.UI.HtmlControls.HtmlContainerControl.Render(HtmlTextWriter writer)

at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)

at ASP.default_aspx.__Render__control1(HtmlTextWriter __w, Control parameterContainer)

at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)

at System.Web.UI.Page.Render(HtmlTextWriter writer)

at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)

at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)

at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)

at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

at Microsoft.Exchange.Management.ControlPanel.WebServiceReference.GetList(DDIParameters filter, SortOptions sort)

at Microsoft.Exchange.Management.ControlPanel.OrganizationCache.LoadTargetDeliveryDomain(AddValueHandler addValue, LogErrorHandler logError)

at Microsoft.Exchange.Management.ControlPanel.OrganizationCache.TryGetValue[T](String key, T& value)

at Microsoft.Exchange.Management.ControlPanel._Default.RenderMetroTopNav()

at ASP.default_aspx.__RendermainForm(HtmlTextWriter __w, Control parameterContainer)

at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)

at System.Web.UI.HtmlControls.HtmlForm.RenderChildren(HtmlTextWriter writer)

at System.Web.UI.HtmlControls.HtmlContainerControl.Render(HtmlTextWriter writer)

at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)

at ASP.default_aspx.__Render__control1(HtmlTextWriter __w, Control parameterContainer)

at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)

at System.Web.UI.Page.Render(HtmlTextWriter writer)

at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)

at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

System.Data.DuplicateNameException: A column named 'TargetDeliveryDomain' already belongs to this DataTable.

at System.Data.DataColumnCollection.RegisterColumnName(String name, DataColumn column)

at System.Data.DataColumnCollection.BaseAdd(DataColumn column)

at System.Data.DataColumnCollection.AddAt(Int32 index, DataColumn column)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase.CreateColumn(DataTable table, Dictionary`2 rbacMetaData)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase..ctor(Service profileBuilder)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase..ctor(String schemaFilesInstallPath, String schema)

at Microsoft.Exchange.Management.DDIService.WSListDataHandler..ctor(String schemaFilesInstallPath, String resourceName, String workflowName, DDIParameters parameters, SortOptions sortOptions)

at Microsoft.Exchange.Management.DDIService.DDIServiceHelper.GetListCommon(DDIParameters filter, SortOptions sort, Boolean forGetProgress)

at System.Data.DataColumnCollection.RegisterColumnName(String name, DataColumn column)

at System.Data.DataColumnCollection.BaseAdd(DataColumn column)

at System.Data.DataColumnCollection.AddAt(Int32 index, DataColumn column)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase.CreateColumn(DataTable table, Dictionary`2 rbacMetaData)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase..ctor(Service profileBuilder)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase..ctor(String schemaFilesInstallPath, String schema)

at Microsoft.Exchange.Management.DDIService.WSListDataHandler..ctor(String schemaFilesInstallPath, String resourceName, String workflowName, DDIParameters parameters, SortOptions sortOptions)

at Microsoft.Exchange.Management.DDIService.DDIServiceHelper.GetListCommon(DDIParameters filter, SortOptions sort, Boolean forGetProgress)

Flight info: Features:[[Global.DistributedKeyManagement, False],[Global.GlobalCriminalCompliance, False],[Global.MultiTenancy, False],[Global.WindowsLiveID, False],[Eac.AllowMailboxArchiveOnlyMigration, True],[Eac.AllowRemoteOnboardingMovesOnly, False],[Eac.BulkPermissionAddRemove, True],[Eac.CmdletLogging, True],[Eac.CrossPremiseMigration, False],[Eac.DevicePolicyMgmtUI, False],[Eac.DiscoveryDocIdHint, False],[Eac.DiscoveryPFSearch, False],[Eac.DiscoverySearchStats, False],[Eac.DlpFingerprint, False],[Eac.EACClientAccessRulesEnabled, False],[Eac.GeminiShell, False],[Eac.ManageMailboxAuditing, False],[Eac.ModernGroups, False],[Eac.Office365DIcon, False],[Eac.OrgIdADSeverSettings, False],[Eac.RemoteDomain, False],[Eac.UCCAuditReports, False],[Eac.UCCPermissions, False],[Eac.UnifiedAuditPolicy, False],[Eac.UnifiedComplianceCenter, False],[Eac.UnifiedPolicy, False],[Eac.UnlistedServices, False],], Flights:[], Constraints:[[LOC, EN-US],[MACHINE, EX2K1301],[MODE, ENTERPRISE],[PROCESS, W3WP],[USER, Username(EditeD)@],[USERTYPE, BUSINESS],], IsGlobalSnapshot: False

Solution:

Looking in the path I have wrote above “\Microsoft\Exchange Server\V15\ClientAccess\ecp\DDI” and opening the file that I have mentioned “RemoteDomains.xaml” you can clearly see there are incorrect format that have been duplicated 3 times.

clip_image003

To make sure that this was the cause, I have another server with CU8 on it so I went and checked the same file which was in the same location to find the result different.

This is the server that works in another environment and doesn’t have any issue.

clip_image004

So, the solution was to remove the two duplicates and correct the format of the variable line… I corrected the first line that includes

<Variable DataObjectName=”RemoteDomain” Name=”TargetDeliveryDomain” Type=”{x:Type s:Boolean}” />

And deleted the other two identical lines.. then I saved the file and closed notepad.

clip_image005

Next: Open IIS on the same server and go to “Application Pools” right click on the affected pools and Recycle them.. You don’t need to reset IIS as the fix should work right away.

Recycle ECP Pool

clip_image006

After recycling checking if the pool is reporting started or not…

clip_image007

Next I open ECP and it works without an issue

clip_image008

Exchange 2010 to 2013 Migration fails with “You cannot have ArchiveDomain set when archive is not enabled for this user”

You cannot have ArchiveDomain set when archive is not enabled for this user.

I have previously done a Hybrid integration with Office 365 with my Exchange 2010 server and enabled Archiving online when I migrated my user to Exchange online but then I finished my demo and decided to bring the user back on-premises.

Now I have deployed Exchange 2013 and wanted to migrate the same user to Exchange 2013 from 2010 but the migration request fails with the following message.

clip_image001

​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​6/7/2015 1:23:24 PM [EXCH2K13] ” created move request.6/7/2015 1:23:57 PM [EXCH2K13] The Microsoft Exchange Mailbox Replication service ‘EXCH2K13.demotesas.local’ (15.0.1076.6 caps:1FFF) is examining the request.6/7/2015 1:23:59 PM [EXCH2K13] Connected to target mailbox ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’, database ‘Mailbox Database 0439787427’, Mailbox server ‘EXCH2K13.demotesas.local’ Version 15.0 (Build 1076.0).6/7/2015 1:23:59 PM [EXCH2K13] Connected to source mailbox ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’, database ‘Database1’, Mailbox server ‘EXCH01.demotesas.local’ Version 14.3 (Build 174.0).6/7/2015 1:23:59 PM [EXCH2K13] Request processing started.6/7/2015 1:23:59 PM [EXCH2K13] Source mailbox information:Regular Items: 104, 5.549 MB (5,818,789 bytes)Regular Deleted Items: 0, 0 B (0 bytes)FAI Items: 50, 0 B (0 bytes)FAI Deleted Items: 0, 0 B (0 bytes)6/7/2015 1:23:59 PM [EXCH2K13] Cleared sync state for request b6ee5dd7-beab-45a0-9933-8e926a694de3 due to ‘CleanupOrphanedMailbox’.6/7/2015 1:23:59 PM [EXCH2K13] Mailbox signature will not be preserved for mailbox ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’. Outlook clients will need to restart to access the moved mailbox.6/7/2015 1:24:04 PM [EXCH2K13] Stage: CreatingFolderHierarchy. Percent complete: 10.6/7/2015 1:24:05 PM [EXCH2K13] Initializing folder hierarchy from mailbox ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’: 76 folders total.6/7/2015 1:24:05 PM [EXCH2K13] Folder creation progress: 0 folders created in mailbox ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’.6/7/2015 1:24:10 PM [EXCH2K13] Folder hierarchy initialized for mailbox ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’: 75 folders created.6/7/2015 1:24:10 PM [EXCH2K13] Stage: CreatingInitialSyncCheckpoint. Percent complete: 15.6/7/2015 1:24:10 PM [EXCH2K13] Initial sync checkpoint progress: 0/76 folders processed. Currently processing mailbox ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’.6/7/2015 1:24:12 PM [EXCH2K13] Initial sync checkpoint completed: 66 folders processed.6/7/2015 1:24:12 PM [EXCH2K13] Stage: LoadingMessages. Percent complete: 20.6/7/2015 1:24:14 PM [EXCH2K13] Messages have been enumerated successfully. 154 items loaded. Total size: 5.55 MB (5,819,724 bytes).6/7/2015 1:24:14 PM [EXCH2K13] Stage: CopyingMessages. Percent complete: 25.6/7/2015 1:24:14 PM [EXCH2K13] Copy progress: 0/154 messages, 0 B (0 bytes)/5.55 MB (5,819,724 bytes), 55/76 folders completed.6/7/2015 1:24:58 PM [EXCH2K13] Copying messages is complete. Copying rules and security descriptors.6/7/2015 1:25:04 PM [EXCH2K13] Initial seeding completed, 154 items copied, total size 5.55 MB (5,819,724 bytes).6/7/2015 1:25:04 PM [EXCH2K13] Stage: IncrementalSync. Percent complete: 95.6/7/2015 1:25:05 PM [EXCH2K13] Folder hierarchy changes reported in source ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’: 2 changed folders, 0 deleted folders.6/7/2015 1:25:05 PM [EXCH2K13] Content changes reported for mailbox ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’: Batch 1, New 3, Changed 1, Deleted 0, Read 0, Unread 0, Total 4.6/7/2015 1:25:05 PM [EXCH2K13] Total content changes applied to mailbox ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’: New 3, Changed 1, Deleted 0, Read 0, Unread 0, Skipped 0, Total 4.6/7/2015 1:25:05 PM [EXCH2K13] Incremental Sync ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’ completed: 2 hierarchy updates, 4 content changes.6/7/2015 1:25:05 PM [EXCH2K13] Stage: IncrementalSync. Percent complete: 95.6/7/2015 1:25:07 PM [EXCH2K13] Final sync has started.6/7/2015 1:25:07 PM [EXCH2K13] Folder hierarchy changes reported in source ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’: 0 changed folders, 1 deleted folders.6/7/2015 1:25:07 PM [EXCH2K13] Incremental Sync ‘b6ee5dd7-beab-45a0-9933-8e926a694de3 (Primary)’ completed: 1 hierarchy updates, 0 content changes.6/7/2015 1:25:07 PM [EXCH2K13] Source mailbox information:Regular Items: 108, 5.562 MB (5,832,087 bytes)Regular Deleted Items: 0, 0 B (0 bytes)FAI Items: 50, 0 B (0 bytes)FAI Deleted Items: 0, 0 B (0 bytes)6/7/2015 1:25:07 PM [EXCH2K13] Stage: FinalIncrementalSync. Percent complete: 95.6/7/2015 1:25:09 PM [EXCH2K13] Mailbox store finalization is complete.6/7/2015 1:25:09 PM [EXCH2K13] SessionStatistics updated.6/7/2015 1:25:09 PM [EXCH2K13] Verifying mailbox contents…6/7/2015 1:25:10 PM [EXCH2K13] Mailbox contents verification complete: 66 folders, 157 items, 5.562 MB (5,831,953 bytes).6/7/2015 1:25:10 PM [EXCH2K13] Mailbox ‘Mohammed JA. Hamada’ was loaded from domain controller ‘ad.demotesas.local’.6/7/2015 1:25:18 PM [EXCH2K13] Fatal error UpdateMovedMailboxPermanentException has occurred.

On Exchange 2010, I launched Exchange Management shell and ran the following cmdlet which will show any attribute that has arch in it for the user Mohammed

Get-mailbox User | fl arch*

clip_image002

Since there’s no archive mailbox then the archive domain is invalid and I don’t even own it anymore as it has expired a while ago.

Resolution:

I will try to remove the archive domain object from the user’s properties using the following cmdlet

Set-mailbox mailboxname -ArchiveDomain $null

clip_image003

Using the above cmdlet seems to fail due to this property being administered by Exchange server so it’ll have to be removed manually.

I will open the user’s attribute and delete the value and try to continue the migration again.

clip_image004

I’ll click on Edit then Clear and OK

clip_image005

clip_image006

clip_image007

Migration finished successfully

clip_image008

clip_image009

Exchange 2010 Performance counters for the Client access role is not installed

Performance counters for the Client access role is not installed

To solve the problem

  • Open the Exchange Management Shell
  • Run the following cmd: add-pssnapin Microsoft.Exchange.Management.PowerShell.Setup
  • Run the following cmd: new-perfcounters –definitionfilename “C:\Program Files\Microsoft\Exchange Server\V14\Setup\Perf\RpcClientAccessPerformanceCounters.xml”

By running these cmds we will install the Performance Counters needed for the RPC Client Access Service. Once installed the error won’t be displayed anymore.

Prepare Schema for Exchange 2013 Migration while having Hybrid Integration with Exchange 2010

In a very interesting situation that I came through I had an environment with two DCs and Exchange 2010 that I had previously setup for Hybrid integration with Office 365 for demonstration with a trial subscription but I haven’t removed the integration after I finished my test and the trial expired and the tenant was deleted.

Next I intended to upgrade my existing Exchange 2010 to Exchange 2013 and setup coexistence between them however, I have stumbled in the step of preparation of AD schema for Exchange 2013. While trying to prepare the schema I got the following error

clip_image001

Setup /PrepareSchema /IAcceptExchangeServerLicenseTerms

Welcome to Microsoft Exchange Server 2013 Cumulative Update 8 Unattended Setup

Copying Files…

File copy complete. Setup will now collect additional information needed for

installation.

Performing Microsoft Exchange Server Prerequisite Check

Prerequisite Analysis FAILED

A hybrid deployment with Office 365 has been detected. Please ensure that you are running setup with the /TenantOrganizationConfig switch. To use the TenantOrganizationConfig switch you must first connect to your Exchange Online tenant via PowerShell and execute the following command: “Get-OrganizationConfig | Export-Clixml -Path MyTenantOrganizationConfig.XML”. Once the XML file has been generated, run setup with the TenantOrganizationConfig switch as follows “/TenantOrganizationConfig MyTenantOrganizationConfig.XML”.

If you continue to see this this message then it indicates that either the XML file specified is corrupt, or you are attempting to upgrade your on-premises Exchange installation to a build that isn’t compatible with the Exchange version of your Office 365 tenant. Your Office 365 tenant must be upgraded to a compatible version of Exchange before upgrading your on-premises Exchange installation. For

more information, see: http://go.microsoft.com/fwlink/?LinkId=262888

For more information, visit: http://technet.microsoft.com/library(EXCHG.150

)/ms.exch.setupreadiness.DidTenantSettingCreatedAnException.aspx The Exchange Server setup operation didn’t complete. More details can be found in ExchangeSetup.log located in the <SystemDrive>:\ExchangeSetupLogs folder.

The Office 365 Hybrid setup was still there in my Exchange Console and since I couldn’t follow MS’s recommended steps to connect to O365 tenant and get the XML file then I had to do things manually.

First I connected to the EMC and removed all the instances that were created during the Exchange Hybrid Wizard Configuration

1- Removing Organization Relationships

clip_image001[4]
image

2- Removing Federation Trust

clip_image001[6]
clip_image002

3- Removing Remote Domains

clip_image001[8]
clip_image002[4]

4- Removing Accepted Domains

clip_image001[10]
clip_image002[6]

5- Removing Send and Receive Connectors

clip_image001[12]
clip_image002[8]
clip_image003

6- Lastly the Hybrid Configuration object…

Since remove-hybridconfiguration cmdlet is not supported to remove the hybrid configuration object from AD then we have no choice but to use ADSIEDIT tool to do so.

I will navigate to Configuration > Services > Microsoft Exchange > First Organization > Delete “CN=Hybrid Configuration”

image
image
image

Restart MSExchangeServicehost

clip_image001[14]
image

Now I will try again to prepare AD schema for Exchange 2013 but I got a different error

clip_image001[16]

Extending Active Directory schema FAILED

The following error was generated when “$error.Clear);

install-ExchangeSchema -LdapFileName ($roleInstallPath + “Setup\Data\”+$

RoleSchemaPrefix + “schema0.ldf”)” was run: “Microsoft.Exchange.Configuration.Tasks.TaskException: There was an error while running ‘ldifde.exe’ to import the schema file ‘C:\Windows\Temp\ExchangeSetup\Setup\Data\PostExchange2003_schema0.ldf’. The error code is: 8224. More details can be found in the error file: ‘C:\Users\Administrator.DEMOTESAS\AppData\Local\Temp\2\ldif.err’at Microsoft.Exchange.Configuration.Tasks.Task.ThrowError(Exception exception, ErrorCategory errorCategory, Object target, String helpUrl)at Microsoft.Exchange.Management.Deployment.InstallExchangeSchema.ImportSchem

aFile(String schemaMasterServer, String schemaFilePath, String macroName, StringmacroValue, WriteVerboseDelegate writeVerbose)at Microsoft.Exchange.Management.Deployment.InstallExchangeSchema.InternalPro

cessRecord()at Microsoft.Exchange.Configuration.Tasks.Task.<ProcessRecord>b__b()at Microsoft.Exchange.Configuration.Tasks.Task.InvokeRetryableFunc(String funcName, Action func, Boolean terminatePipelineIfFailed)”.The Exchange Server setup operation didn’t complete. More details can be found

in ExchangeSetup.log located in the <SystemDrive>:\ExchangeSetupLogs folder.

Checking the ldif.err file mentioned in the error above it seems that Exchange is complaining because the changes of the AD schema is not being replicated to the other AD partners which is true since I have another additional DC that’s turned off.

clip_image001[19]

After turning on the other DC we’ll see what happens

The other DC had another issue as I have turned it off for long time and it was not syncing due to expired Tomb stone life so I had to fix this issue as well and I have published it in a different article.

Please click here to see how the replication issue was fixed.

Issue has been fixed

clip_image001[21]
clip_image002[10]

Hope someone finds this useful

Exchange 2010 Console after DC migration stopped working

After DC migration and changing in the PDC and Schema master role server to the new DC and shut down the old DC for test. On Exchange 2010 server you might get the following error

An error caused a change in the current set of Active Directory settings. Restart The Exchange Management Console.

Exchange Console

clip_image019

Current deployment

  1. Exchange 2010
  2. New DC 2012 R2 with another Additional DC installed newly.
  3. Two DC 2008R2 but have been shut down for testing.

Problem:

After you shutdown or demote the old PDC or Schema master Demote Domain Controller role, Microsoft Exchange Management Console fails to retrieve any Exchange information with error message “An error caused a change in the current set of Active Directory Server settings. Restart Exchange Management console.”

Cause

Microsoft Exchange management console caches the data in the user’s profile for quick access, So whenever you try to open EMC from an existing Exchange admin profile you will get the same error.

Resolution:

Navigate to the following folder and delete the Exchange Management Console file.

%userprofile%\appdata\roaming\Microsoft\MMC\Exchange Management Console

clip_image020

Close EMC and reopen it and you should be done.

Exchange 2007/2010 Doesn’t show new DC (2012) servers after adding them as additional DCs

Symptoms

In an environment where one DC exist after adding Windows 2012 R2 Servers as additional servers, Exchange 2007 doesn’t show the new servers although they also hold GC.

image

Research:

To locate the problem you should search the event ID (2080) which shows the populated DCs and the permissions allowed on Exchange servers

In the below screenshot, the SACL right was not provided to the new DCs due to GPO problem.

image

After checking sites, Replication, all is healthy and no issue with it.

3 servers (Two 2012 servers) and one DC 2003 Server

Exchange 2010 SP3 servers.

image

Reason:

The Default Domain Controllers Policy was not linked to the Domain Controllers OU.

image
image
image

Resolution:

After Linking the Domain Controllers OU to the Default Controllers policy, the SACL permission was provided without any issue.

image

Now Exchange is reporting healthy and can read the new DCs which allow us to demote the old DCs

image

After removing the old DC

image

Upgrade Microsoft Domain Controller 2008 R2 to DC 2012 R2 with Exchange 2010 in the current environment.

Upgrade Microsoft Domain Controller 2008 R2 to DC 2012 R2 with Exchange 2010 in the current environment.

Prerequisites:

1- Windows 2012 R2 fully patches

2- New Windows 2012 R2 server should be joined to the Domain controller 2008r2

After you get all the prerequisites ready, start the Server manager and click on Add roles then add the ADDS role and follow the following instructions

Install the role and the n configure it as following

clip_image001

Add it to the existing DC

clip_image002
clip_image003
clip_image004
clip_image005
clip_image006
clip_image007
clip_image008
clip_image009
clip_image010

To migrate the AD Operations Master roles. The simplest way to move these roles is via PowerShell.

On Server 2012 AD PowerShell modules, this can be done from anywhere. Simply run the following command to view you current configuration, and change them:

PS C:\> netdom query FSMO

clip_image011

Move-ADDirectoryServerOperationMasterRole -identity “dc1” -OperationMasterRole 0,1,2,3,4

clip_image012
clip_image013

Making sure that all the roles have been migrated :

netdom query FSMO

clip_image014
clip_image015

Adding second DC

clip_image016

Reference:

https://technet.microsoft.com/en-us/library/ee617229.aspx?f=255&MSPPError=-2147217396

Source: Default-First-Site-Name\DC2

******* 1 CONSECUTIVE FAILURES since 2015-03-23 19:37:45

Last error: 8524 (0x214c):

The DSA operation is unable to proceed because of a DNS lookup failu

re.

Naming Context: CN=Configuration,DC=domain,DC=local

Source: Default-First-Site-Name\DC2

******* WARNING: KCC could not add this REPLICA LINK due to error.

Naming Context: CN=Schema,CN=Configuration,DC=domain,DC=local

Source: Default-First-Site-Name\DC2

******* WARNING: KCC could not add this REPLICA LINK due to error.

Naming Context: DC=domain,DC=local

Source: Default-First-Site-Name\DC2

******* WARNING: KCC could not add this REPLICA LINK due to error.

clip_image017

Resolution:

After joining new DC you will see this error until the replication with the PDC and schema master is finished.

Use the repadmin /syncall to hasten the sync process.

clip_image018

After we changed the PDC and Schema master role server to the new DC and shut down the old DC for test. On Exchange 2010 server you might get the following error

Exchange Console

clip_image019

Current deployment

  1. Exchange 2010
  2. New DC 2012 R2 with another Additional DC installed newly.
  3. Two DC 2008R2 but have been shut down for testing.

Problem:

After you shutdown or demote the old PDC or Schema master Demote Domain Controller role, Microsoft Exchange Management Console fails to retrieve any Exchange information with error message “An error caused a change in the current set of Active Directory Server settings. Restart Exchange Management console.”

Cause

Microsoft Exchange management console caches the data in the user’s profile for quick access, So whenever you try to open EMC from an existing Exchange admin profile you will get the same error.

Resolution:

Navigate to the following folder and delete the Exchange Management Console file.

%userprofile%\appdata\roaming\Microsoft\MMC\Exchange Management Console

clip_image020

Hope this was useful

Restoring an Active Directory Object after mistakenly deleting it

Active Directory Recycle Bin

Starting with Windows 2008 R2, Microsoft introduced the Active Directory recycling bin. This is great for recovering objects back into AD if they are accidentally deleted. In order to use the recycle bin feature, your forest must be running with a functional level of Windows 2008 R2. If your forest is running at this level you simply run a PowerShell command to enable it.

Enable

To enable Active Directory Recycle Bin using the Enable-ADOptionalFeature cmdlet

Click Start, click Administrative Tools, right-click Active Directory Module for Windows PowerShell, and then click Run as administrator.

Below is a sample for enabling it for moh10ly.com:

Enable-ADOptionalFeature –Identity “CN=Recycle Bin Feature,CN=Optional Features,CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuration,DC=moh10ly,DC=com” –Scope ForestOrConfigurationSet –Target moh10ly.com

clip_image001

Restore

Once you have the Recycling Bin for Active Directory you will have to use LDP.exe to restore. By default the container with the deleted objects is not displayed. The following steps will allow you to see the container with the deleted objects.

Display Deleted Objects

Follow these steps to display the Deleted Objects container:

  1. To open Ldp.exe, click Start, click Run, and then type exe.
  2. On the Optionsmenu, click Controls.
clip_image002

3. In the Controlsdialog box, expand the Load Predefined pull-down menu, click Return deleted objects, and then click OK.

clip_image003

4. To verify that the Deleted Objects container is displayed:

  • To connect and bind to the server that hosts the forest root domain of your AD DS environment, under Connections, click Connect, and then Bind. (U must use SSL and port 636)
  • Click View, click Tree, and in BaseDN, type DC=<mydomain>,DC=<com>, where <mydomain>and <com> represent the appropriate forest root domain name of your AD DS environment.
  • In the console tree, double-click the root distinguished name (also known as DN) and locate the CN=Deleted Objects, DC=<mydomain>,DC=<com>container, where <mydomain>and <com> represent the appropriate forest root domain name of your AD DS environment.
clip_image004
clip_image005
clip_image006

Restore Deleted Objects

Once you have enabled the container to be displayed, you can now restore deleted objects from Active Directory. Below are the steps to recover a single item from the recycle bin using LDP.exe.

Follow these steps to restore a deleted Active Directory object using Ldp.exe:

  1. Open Ldp.exe from an elevated command prompt. Open a command prompt (Cmd.exe) as an administrator. To open a command prompt as an administrator, click Start. In Start Search, type Command Prompt. At the top of the Startmenu, right-click Command Prompt, and then click Run as administrator. If the User Account Control dialog box appears, enter the appropriate credentials (if requested), confirm that the action it displays is what you want, and then click Continue.
  2. To connect and bind to the server that hosts the forest root domain of your AD DS environment, under Connections, click Connect, and then click Bind.
clip_image007

3. On the Options menu, click Controls.

4. In the Controls dialog box, expand the Load Predefined drop-down list, click Return Deleted Objects, and then click OK.

5. In the console tree, navigate to the CN=Deleted Objects

clip_image008

6. Locate and right-click the deleted Active Directory object that you want to restore, and then click Modify.

7. In the Modifydialog box.

8. In Edit Entry Attribute, type isDeleted.

9.Leave the Valuesbox empty.

10. Under Operation, click Delete, and then click Enter.

clip_image009
clip_image010

11. In Edit Entry Attribute, type distinguishedName.

12. In Values, type the original distinguished name (also known as DN) of this Active Directory object.

13. Under Operation, click Replace.

clip_image011

14. Make sure that the Extended check box is selected, click Enter, and then click Run.

clip_image012

A key point to understand and remember with AD Recycle Bin is that you must restore hierarchically; a parent object must be restored before a child object. If you were to delete an entire OU and all its contents, you must first restore the OU before you can restore its contents.

Modify

clip_image013

Clicking on Run gives an error

“Error 0x2077 Illegal modify operation. Some aspect of the modification is not permitted.”

clip_image014

Resolution:

Disconnect and reconnect with SSL on port 636

clip_image015

Enter the full Distinguished path in the Values

clip_image016

Click on Run again and that should work

clip_image017

Before

clip_image018

After

clip_image019

\

After restoring the object, I will try to login to the user’s mailbox

I’ll need to reset the user’s password after its restored.

Time to login

Resetting Usernames and Passwords from text file

I had a project to migrate users from Linux Postfix mail system to Exchange 2013 but had to do it in stages in order to use it as recommended by Microsoft, The customer had Red Hat Linux Servers with Postfix integrated with Active directory for authentication.

In order to migrate the users I installed Exchange 2007 in order to use the Microsoft tool called (Microsoft Transporter Suite) and had to reset all users (850 User) passwords to use notepad file and import it in the tool in order to migrate all users in less than a week.

I searched for any powershells that would reset users passwords on Active directory but could not find anyone that would suit my scenario and customer’s security policies until I came through a tool called “Quest One activeRoles which integrates lots of useful commands into its own powershell that have to be installed on Active Directory server to reset all users passwords.

You can find the tool either from the following link:

http://www.quest.com/powershell/activeroles-server.aspx

In order to do so I have prepared a notepad file with 2 columns “Username, password” and then copied all users and their passwords below the two columns, saved the file in .csv extension then used the following script

$data = Import-csv “C:\users_pass.csv”

foreach($line in $data) {set-QADUser $line.username -UserPassword $line.password }

Click on the snapshot to enlarge it

As shown in the snapshopt above you will have to install the application first on your DC and then run the application shell “ActiveRoles Management shell for AD” as administrator then run the commands below

$data = Import-csv “C:\users_pass.csv”

foreach($line in $data) {set-QADUser $line.username -UserPassword $line.password }

Note:

Usernames in the CSV File must be according to the format in the Snapshot above or else the command won’t be recognized.