Wednesday, May 27, 2009

quick summary of how exchange search works

fixing search indexing problems:

Method 1: Re-register MapiProtocolHandlerStub.dll
a. Register MapiProtocolHandlerStub.dll
regsvr32 MapiProtocolHandlerStub.dll
b. Stop the msexchangesearch service
c. Delete the catalog folder
d. Restart the msexchangesearch service
Method 2: remove and re-install the Microsoft Full Text Indexing Engine, please follow the answer of Elvis Wei in this thread

more here: http://social.technet.microsoft.com/Forums/en-US/exchangesvrgeneral/thread/6b3f9dcb-f97f-4e89-aade-644d6494a91c

Thursday, May 14, 2009

Microsoft Exchange Server 2010 Transition and deployment - UNC 318

A. Introduction:
1. New form 2007 to 2010:
· Run exchange server yourself, or use exchange online
· High availability solution for mailboxes is database availability group (DAG)
a. Also provides site reillience3 and dr
b. 30sec failover with simplified admin experience
c. Works with cheap disk
d. Replaces SCR, LCR, SCC and CCR from 2007
· EMC (Exchange mangement console) client access server
a. Improved high availability solution: outlook MAPI connects directly to CAS
B. Transition:
1. In nutshell:
· Transition ad site by ad site
· Start with internet facing AD sites
· Upgrade all your existing servers to SP2
· Deploy 2010 servers CAS first, MBX last. Start with few and gradually add more servers as you move mailboxes
2. Steps:
· Legacy hostname for old FE/CAS
· SSL cert purchased
· End users don’t’ see this host mane
· Used when auto discover and redirection form cas2010 tell clients to talk to fe2003 /cas 2007 for MBX 2003 and MBX 2007 access
3. Move:
· Internet host manes to cas2010
· Um phone number to um 2010
· SMTP end point to hub2010
·
4. Move mailboxes
5. Decommission old servers
6. Move to next AD site
C. Prerequisites:
1. Windows server 2008 sp2 or r264 bit, standard, enterprise...
2. Client supported , outlook 2007, enterouge 2008, outlook 2010

D. Tools for migration:
1. Remote connectivity analyzer:
· https://www.testexchangeconnectivity.com
· In beta
· Features: Active sync connectivity test
· Use this tool validate any change
E. Migration/Deployment:
1. Setup:
· Step by step instructions in setup app
· Setup.exe with parameters gives unattended setup
· Try it this week – UNC14-HOL
· Configure 2010:
a. Your preferred configuration
b. External cas hostname: mail.andaker.com. exernalURL parameters
2. Certificates and SSL
· Best practice: minimize the number of certificates:
a. 1 certificate for all CAS serves + reverse proxy + edge/hub
b. Use ‘subject alternative name” (SAN) certificate which can cover multiple hostnames
c. Don’t list machines hostnames in certificated hostname list: use load balance (LB) array
d. Minimize the number of host names
3. CAS load balancing:
· Best practice: use ‘split dns” for exchange hostnames used by clients. Goal: minimize number of hostnames
a. Mail.andaker.com for exchange connectivity on intranet and internet
b. Mail.andaker.com has different IP addressing intranet/internet DNS
· Here’s the gotcha: OWA and EWS load balancing require client <-> sever affinity:
a. Client –IP based window NLB or LB device using cookie based affinity
· Tell autodiscover where to send clients : configure internal URL and externalURL parameters on virtual directories:
a. E.g. Set-owavirtual directory
4. Certificates: step by step
· New-exchange Certificate*
· You can also use certificate wizard
a. Takes you through list of questions to list all host names you need
b. The rtm version will include configuration for legacy CAS
5. Switching to 2010 CAS:
· configure reverse proxy (eg.isa) or external dns to point legacy.anddaker.com to fe2003/cas2007
· transition from 2003: ensure owa can redirect user to correct URL: configure exchange 2003 url parameter on CAS2010 owa virtual directory
· test before the plunge: legacy.andakercom works for internet access
6. the plunge:
· transition from 2007: tell CAS2010 how to send users to CAS2007:
a. configure externalURL parameter on cas2007 virtual directories (OWA, EAS, EWS, OAB..) to point to legacy.andake.com
b. tests that cas 201o is redirecting
· configure reverse proxy or dns:
a. switch mail.candakar.com to go to 2010 cas and edge/hub
b. users will start using 2010
7. demo:
· setting the virtual directory on 2010 to point exchange 2003 users to right directories by use of -external2003URL
· view from ISA server: publishing 2010 rules on ISA:
a. you can go to paths and create new path mappings for 2010 (e.g., /ews/*, /autodiscover)
b. update the DNS records pointing to 2007 to 2010
· run test exchange connectivity
· you get the logon page of 2010 but it redirects to owa 2003
8. client access transitions:
· client access cas2010
· four different things happen for 2003/2007 mailboxes
a. autodiscover tells clients to talk to cas 2007
b. http redirect to FE (front end 2003 or CAS 2007
c. praying of requires form CAs 2010 to CAS 2007: for pop and imap there is no redirection
d. direct CAS 2010 support for the service against BE 2003 and MBX 2007
9. smtp transport transition:
· upgrade existing servers to sp2
· introduce 2010 servers
· switch edge DNS + stmp to go to hub2010, you can continue to use 2007 edge just change out the subscription
10. Exchange UM:
· Introduce 2010 UM
· Add the UM to dial plan you already have
· Configure gateway
· You tell everyone to talk to 2010 first. 2010 is smart enough to redirect users on 2007 to 2007 UM
11. Exchange UM: OCS
· You need to add new dial plan\
· When you move user mailbox you also need to add user to new dial plan
12. Mailbox moves:
· 1 Gb mailbox could take 90 minutes to move:
a. Pain: user is disconnected for duration
b. Pain: your SLA for availability is not met
· Online move mailbox:
a. new feature in 2010
b. users remain online while mailboxes are moved: at end of move user is briefly disconnected as recently received messages are copied over, client autodiscover new database location
c. admins can perform migration and maintenance during regular hours
d. online: 2007, 2010 -> e2010 exchange online
e. offline: 2003-> 2010
f. commandlets available for mailboxes very powerful, it also gives reporting and granularity for the details of move
13. mailbox move demo:
· option to move local and remote (hosted service)
· you can track move requests and their status



Exchange 2010 High Availability

A. Intro
1. Solution delivered:
· Unified technology for high availability and site resilience
· New framework for creating highly available mailboxes
· Evolution of continuous replication technology
· Can be deployed on range or storage options
· Native to exchange not bolted onto the side; no longer uses cluster model but HA is built in
B. Legacy versions:
1. Exchange 2003 server:
· Challenges were using cluster tools/ resources to manage exchange
· Failover was always at the server level
· 3rd party site resilience
2. Exchange 2007:
· Clustering only allows mailbox
· You need clustering knowledge to deploy mailbox CCR/SCR
C. What’s new in Exchange 2010:
· Database replication at the DB level
· On 2007 replication was at SG level
· You can choose replication target where as on 2007 it was automatically done
· Failover is at database level, exchange manages failover
· All clients connect to RPC client access server. CAS server knows which mailbox server to connect to client access
· You can have mailbox server in different sites and still replicate
1. Feature names;
· Mailbox resiliency – name of unified HA and site resilience solution
· Database availability Group – a group of up to 16 mailbox server that host a set of replicated database
· Mailbox database copy – the ability to deploy high availability/site resilience after exchange is installed
· Exchange 3rd party replication API – an exchange – provided API that enabled use of 3rd party replication of a DAG in lieu of continue of replication
· New API will be provided for 3rd party could write to so continuous replication can be done by 3rd party
2. Terminology:
· HA – solutions must provide data availability, service availability, and automatic recovery from failures
· Disaster recovery – process used to manually recover from a failure, step you take when HA is no longer there
· Site resilience – DR solution used for recovery from a site failure,
· *over – short for switchover/failover: a switchover is a manual activation of one or m ore database; a failover is an automatic activation of one or more database after a failure
3. Exchange 2010 *over: (could be failover or switchover)
· Within a datacenter: database or server * server
· Datacenter level: switch over
· Between datacenters:
a. Database for server * over’s
b. Assumptions:
- Each datacenter is a separate active directory site
- Each data center has live, active messaging services
- Standby datacenter must be active to support single database *over
4. Exchange 2007 concepts brought forward
· Extensively storage engine: database and log files
· Continuous replications:
a. Log shipping and replay
b. Database deseeding
c. Store service, replication service
5. Not coming over:
· Storage groups
· Database identified by the t server on which they live
· Server names as part of database names
· Clustered mailbox servers:
a. Preinstalling a windows failover cluster
b. Running setup in clustered mode
c. Moving a CMS network identity between servers
d. Shard storage
· Two HA copy limits
· Private and public networks (mapi network and replication network are used)
6. 2010 ha fundamental:
· Database availability group server
· Database
· Database copy
· Active manager
· RPC client access
7. DAG:
· Base component of HA and site resilience
· A group of 16 servers that host a set of replicated db
· Wraps a windows failover cluster
a. Manages membership (DAG member = node)
b. Provide heartbeat of dag members servers
c. Active manager stores data in cluster database
· Defines a boundary for:
a. Mailbox db replication
b. Database server *over
c. Active manager
8. Active manager:
· Brain of HA
· Exchange component that manages *over
a. Runs on every sever in a dag
b. Selects best available copy on failovers
c. Is the definitive source of information on where a database is active:
- Stores this information in cluster database
- Provides the information to other exchange components (e.g. RPC client access and hub transport
· Tow active manger role
· Pam – primary active manager
a. Runs on a node that owns the cluster group
b. Gets topology change notification
c. React to server failures
d. Selects the best db copy on *over
· SAM – standby active manger
a. Runs o n every other node in DAG
b. Responds to queries about which server hosts the active e copy
9. How does it select:
· Active manger select the best copy to become active when existing active fails
· Ignores servers that are unreachable or activation is temporarily or regularly blocked
· Sorts copies by currently to minimize data loss
· Breaks ties during sort based on activation preference
· Selects from sorted listed based on copy status of each copy
10. 10 criteria:
· Phase 1: looks at catalog health, copy status is healthy, copy queue length is <10 and replay is <50
· Phase 2-10: As we move down in phases the criteria’s becomes less strict
11. Example: database failover
· Database failure occurs
· active manager moves acxtitve db
· Datgabase copy is restored
· Similarly within and across datacenter
12. Server failure:
· Server failure occurs
· Cluster notification of node down
· Active manager moves active db
· Service restored
· Cluster notification of node is up
· Db copies resynchronize with active db - no need to reseed, this is all automatic
· Similar flow within across datacenters
13. Dag life cycle:
· Dag is created initially as empty object in ad: continuous replication or 3rd party replication using 3rd party replication mode
· When first mailbox serve is added to a dag
a. A windows failover cluster formed with a node majority quorum using the name of the dag
b. These severs added to the dag objects in ad
c. A cluster network object for the dag is created in the build in computers container
d. One or more ip addresses is assigned to the dag
e. The name and the IP address of the dag is registered in dns
· When second and subsequent mail server is added to the dag
a. The server is joined to the cluster for the dag
· After server has been added to a dag
a. Configure the dag: network encryption and network compression
b. Configure dag networks: network subnets, enable/disabled map traffic recitations
c. Create mailbox db copies; seeding is performed automatically –(manually seeding, used when seeding from passive copy of db, this is now possible with 2010)
d. Monitor health and status of db dopes
e. Perform switch over as needed
· Before you can remove a server a from a dag you must first remove all replicated db from sever
· When a server is removed from a dag
a. The server is evicted from cluster
b. The cluster quorum is adjusted as needed
c. The server is removed from the dag object in ad
· Before you remove a dag you must first removal server form the dag
14. Deploying exchange 2010 HA features:
· Prepare Hw, install OS and update
· Run setup and install mailbox role
· Create a dag and replicate db
· Test and *over
· No need to do the ground work for HA, you can do this later
15. 2010 incremental deployment
· Create a dag: new-databaseavaillabilitygroup – name dag1 –filesharewitnesshare
· Add first mailbox server to dag: add-databaseaavaillabiltiuygorup – name dag1 –fileshrewintesshare
· Add second and subsequent mailbox server to
· Add mailbox database copy
· Extend as needed
16. Demo:
· Db is decoupled from the server and this is evident from EMC
· Activation preference – preferred list of sequence number
· DAG configuration:
a. member server
b. Witness sever and witness path
c. DAG network, collection of subnet that you provide, if you have multiple sites, you'll have subnets here configured to a DAG
· Database activation: (switch over)
a. Test-replicationhealth – this can run remotely
d. Get-mailboxcopystatus (instead of storage group copy status)
b. Switch over before the maintenance
c. Auto database mount configuration: lossless, good availability best availability, best effort, none, these refer to log loss
d. It’s not moving anything, the data is already there, it may be copying some logs

· Owa experience:
a. Seamless, user had to refresh using owa
· Delta between beta and rtm
· Creating a DAG, really quick
· Adding members to a dag: this takes little longer
a. Add dag servers serially, you can multi select as many as you want
· Create a mailbox database copy
a. It does a db seed basically
b. It assigns activation preference
17. Transitioning to exchange 2010 ha
· Verify that you meet the requirement for 2010
· Deploy 2010
· Use 2010 mailbox move feature to migrate
· Unsupported transitions
a. In place upgrade to exchange 2010 from any previous version of exchange
b. Using database portability between exchange 20010 and non exchange 2010
c. Backup and restore of earlier versions of exchange db on exchange 2010
d. Using continuous replication between exchange 2010 and exchange 2007
18. End to end improvement:
· Online move mailbox
a. Supported between exchange 2010 db and between exchange 2007 sp2 and exchange 2010 db
b. User can access their mailbox while move is in progress
c. move is performed asynchronous by new service called the ms exchange mailbox replication service (MRS) which runs a client access servers
d. you can move from anywhere because remote control.
e. Built in throttling and recovery, doesn’t impact replication
· RPC client access server
a. New service that establishes a rpc endpoint for client access on the cas role to replace the existing rpc endpoint on the mailbox role
· Shadow redundancy:
a. Protection for message while it’s in transit against edge failure
b. Shadow queue is kept there until message is delivered
? what about when
· Transport dumpster:
a. Gets feedback from replication pipeline to let it know when to delete items
j- once something has been delivered, and the logs are the messages are replicated, transport dumpsters can delete the message
replay is not required for deleting items from dumpster, only data in dumpster is data that has not yet been replicated. This reduced iops from hub transport
b. Responds to requests for redelivery after lossy failover both within its ad sites and across ad sites (old site and new site)
· Replication for other purpose:
a. Site /server/disk failure
b. If you have at least 3 copies you can use ha for backup
c. Archiving/compliance: e-mail archive provides this
d. Recover deleted items: recovery deleted items retention, you can set this to longer, use the hold policy( this is function of dumpster)
e. No longer need to have copy at storage level (raid)

19. Examples: small office:
a. Hardware load balancer
b. cAS/hub/mbx all running on one server, with two servers
c. no more than 8proc core, no more than 16 gig memory
20. Example: double resilience
a. single site
b. 2 nodes, 3 HA copies
c. Jbod, 3 copies
21. Example: Better option, 4 node dag:
a. Single site
b. You protect against failure during maintenance
c. You can lose two copies and still have quorum
22. Take away
· greater end to end availability mailbox resilience
· Unified frame work for high availably and site resilience
· Faster and easier to deploy with incremental deployment

Storage in Exchange 2010 -UNC321

A. Introduction:
1. Exchange storage background
2. Storage technology 2010+
3. Large mailbox value
4. E2010 storage architecture
a. Storage innovation
b. ESE database innovations
5. E2010 storage design
6. Summary
B. Exchange storage 2003 HA /storage design
1. MSIT 4+3 SCC san example
a. +1 IOPS/mailbox
b. 4 active nodes
c. 3 passive nodes
d. 4 gig ram
e. 4000 users/server
f. 250mb mailboxes
g. Backups: daily full, stream to disk/tape
2. Problem with this example
a. Disk was single point of failure

C. Exchange 2007 HA / storage design
1. MSIT CCR + DAS example
a. .33 IOPS/mailbox
b. ~4000 mailboxes/cluster
c. 8 processor cores
d. 2 GB mailboxes
e. Backups: DPM , 15 min incremental, daily express full
f. Using RAID 5
g. No single point of failure
D. Disk technology:
1. Disk capacity trend predicted to continue
a. 2 TB desktop class sata disk available
b. 1 TB near line/midline SAS disk available
2. Sequential through put increasing linearly based on area density
a. 2010 stat = ~250mb/sec
3. Random IO performance not expected improve substantially
a. 15k rpm
4. Random vs. sequential disk IO
a. Random IO
· Disk head ahs to move to process subsequent IO
· Head movement = high IO latency
· Seek lentency limits IOPS
b. Sequential IO
· Disk head doesn't move to process subsequent IO
· Stationary head = low IO latency
· Disk rpm speed limits IOPS
· 7.2 k SATA disk 20ms latency
5. Flash/SSD : 2010 scenario
a. Flash best utilized by 2010 when used as a cache within storage stack
b. Price delta is huge
E. Email trend
1. Average corporate user today can expect to send and receive about 156 messages a day and this numbers exceeded growth about 233 messages a day
F. Large mailbox value:
1. Expectation is growing for large mailbox
2. Large mailbox = 1=10 GB
a. Aggregate mailbox = primary mailbox + archive mailbox
b. 1 year of mail (minimum)
c. 1 year, 48,000 items, 2,400 MB
3. Increased knowledge worker productivity
a. Reduced mailbox management
b. Client accessibility (owa, outlook, mobile)
4. Eliminate/reduce PST
5. Eliminate reduce 3rd party archive solution
6. Client experience:
a. Outlook 2007 performance in cache mode
· Problem with large OST
· Office 2007 sp2 solves this problem (will allow up to 10gig of OST)
· 2010 archive mailbox to reduce data cached to OST (archive is online only)
· 2010 store /ESE changes improves cache mode sync speed
b. Outlook 2007 online /owa performance
· Item/folder item limitations
· View creation performance
· 2010 store/ese changes will allow up to 100k items
c. Client search performance:
· 2010 search performance improvements: real time result views, 3x increasing indexing performance
7. Large mailbox challenges and solutions
a. Long backup times:
· Backup off passive copies
· Daily incremental/weekly full backup
· DPM express full backups
· 2010 HA + hold policy as your backup*
b. Fast recovery: requirements (RT))
· 2010 HA is the fast recovery solution
c. High storage cost: IPS (efficiently utilizing low performance/high capacity disk ) raid overhead
d. Move mailbox downtime
·
e. Database maintenance
G. 2010 storage vision:
· IO reduction
· Sequential IO
· Large, fast low cost mailboxes: sata/tier 2 disk optimization
· Raid less storage (JBOD)
· Storage design flexibility
H. IOPS reduction: store schema changes
1. Store schema = the way the store organizes SATA in the ese database
2. 2010: one simple theme
· Move away from doing many, random small size disk IO to doing fewer sequential, large size disk IO
3. Significant benefits
4. 2007 schema: store table architecture
a. Mailbox take
b. Folder table
c. Message table
d. Attachment table
e. Message/folder table/(per folder)
· Give benefit of single instance storage
· Great way but causes random access
5. 2010 schema:
· Per database: mailbox table
· Per mailbox: folder table, message header table, body
· Per view: view table (e.g. from)
· Single instance storage is gone completely
6. Store schema changes: physical contiguity
· Few large IOs and sequential reads...
7. Store schema changes: logical contiguity
8. Store schema changes as: lazy view updates: reducing IO defying view updates, view updates utilize sequoia IO
· 2007 : IO when ever new messages come in
· 2010: doesn’t do any IO if you are not accessing
· View updates only happen upon user request
· Reads are sequential so they are fast

9. Demo:
· 2007 user on one side and 2010 on other side looking at 100k items mailbox and opening the mailbox for the first time
· Perfmon is running to capture numbers
· 2007, took about 5 sec
· 2007: doing first time sort, you get the pop up.....
· 2007: looking at IO, you can see that it’s random IO and it took 25 sec to open a view
· 2010: starting outlook, 2.5 sec
· 2010: first time sort of 100k items – 5 sec
· 2010: looking at performon, sequential IO
· You can put more users on disk and IO goes down, better experience
10. IOPS reduction: ese changes
· Optimize for new store schema
a. Allocate dab space in contiguous manner
b. Maintain db continuity over time
c. Utilize space efficiently (db compression)

· Increase IO size
a. Db page size increase from 8kb to 32 kb
b. Improve read/write IO coalescing (gap coalescing)
c. Provide improved asynchronous read capability (pre read)
· Increased cache effectiveness
a. 100 mb checkpoint depth (ha configuration only
b. Db chce
11. IIOPS reduction: space manamgement: allcoate space based contiguity
· Databas space allcation hints
· Allocate db space based on either data compactness or data contiguity ( usage pattern)
12. IPs reduction: maintenance contiguity:
· New db maintenance architecture
· Clean up performed at run t time when hard delta occurs. Happened during dumpster cleanup OLM, pages are zeroed by default
· Space compaction: db is compacted and space reclaimed at run-time. auto throttled
· Maintain contiguity: database is analyzed for contiguity and space at run time and is defragmented in the back ground ( b + tree defrag/old2). Auto throttled. Sacrificing space for contiguity
· Database checksumming: two options (both active and passive copies)
a. Run db checksum during run time
13. IOPS reduction: DB contiguity results:
· Message 2007 message folder table – looks random and fragmented
· Messages header table on 2010 – you see contiguity
14. Mitigate db space growth: database compression:
· Store schema changes, space hints, b+tree defrag & 32kb page size combine to increase db file size by 20%
· Growth is 100% compression for message headers and text/html bodies (long values)
· With the compression we can bring our db size down, with mix of html /txt message
15. IPs reduction: db page size increased to 32kb
· Now we don’t need as many page files and we can fit more per page and it’s contiguous
· When comparing 2007 to 2010, a 20kb message can be read in 1 IO while it took 3 IO in 2007
16. IOPS reduction: IO gap coalescing:
· Read case
· For 2007 3 IO reads to get a message off disk
· For 2010: 1 IO
17. IOPS reduction: 100mb check point depth:
· Checkpoint depth = the amount of data that is waiting to be committed to the db file (Edb)
· 2010 default check point depth max is increasing form 20mb to 1000mb only on db protected by 2010 ha (stand alone is 20mb )
· Deep checkpoint benefit + efficient db writes (~40 % reduction)
· Deep checkpoint risks = long store shutdown times, long crash recovery times:
· Risk mitigation: shutdown dab in parallel, failover on store crash
18. Database cache compression:
· Problem: new store schema + 32kb pages ca n reduce efficient of cache. E.g. a page with 8kb of data consumes 32kb of memory in db cache
· Solution: implement db cache compression to shrink partially used cached pages in memory, allowing more effective cache
· Up to 30% more cache/mailbox server = less DB IO
· More mailboxes you have more benefit.
19. DB cache priority:
· Problem: background and recovery db operations can pollute the cache. E.g. db check summing, old2, ha log replay
· Solution: implement d cache priority to allow lower cache priorities for background/replay operations
· There is competition of past, now, future , and cache eviction on the time line of cache
· HA log replay (passive) will come in as past
· DB maintenance will come in at very tail end of the time line


I. Exchange 2010 storage and feeds:
· Db IO increased by 5x
· Log IO write is the same
· For 3000 mailbox:
a. 70% reduction for DB IO/sec
J. Exchange iops trend:
· 90% storage design change
K. Optimize for sata/tier 2 disks:
1. Db writes IO ‘burstiness’: bursty db writes negatively affects db read and log write latency: the more write IO’s issued at a time, the more disk contention
· Solution: throttle db writes abased on checkpoint target (QoS), DB write smoothing
· Works great on sata 7.2 k disk midline
· Results: 50% reduction in RPC latency due to IO smoothing
L. Putting it all together:
1. 2010 storage improvements cannot be quantified in iops reduction alone
M. Jbod/raid less storage: now an option:
1. Jbod: 1 disk = 1 database/log
2. Requires 2010 HA ( 3 _ db copies)
3. Annual disk failure rate (AFR) ~ 5%
4. Advantages:
· Reducing storage cost
· Eliminate unnecessary db cost: server and storage redundancy can be symmetrical
· Reduce disk IO
· Enable simpler storage dsign 1 disk = 1 db
· Enable simple storage failure recovery
5. DisadvantAGES
· Disk stipp8ng performance cannot be leveraged
· Disk failures – db failover
· Re-enabling resiliency = spare disk assignment/partitioning/format/db reseed
· Soft disk errors: bad blocks must be detected and repaired
6. 2010 optimization”
· Improve ha storage failures
· Optimize ha
· Improve storage failures detection (bad blocks /corruption)
a. Active/passive copy background scan
b. Active passive copy lost write detection
· Improve db seeding repair
a. Utilize db passive copy for seeding source
b. Seed capability for content index catalog
7. Reduce reseed by using single page restore (active and passive)page
· Page corruption detection on active copy
· Active db places marker in log stream to notify passive copies to ship up to date page
· Passive receives log and replays up to marker, retrieves good page involves replay service callback

Wednesday, May 13, 2009

Huge changes on storage schema for exchanges. On 2010 single store instance is removed in favor of disk contiguity and sequential writes!

Nifty little tool to test Exchange CAS stuff

https://www.testexchangeconnectivity.com/

Exchange 2010 Management tool - UNC318

A. Introduction:
1. 2010 is 64 bit only
· Admin tools also require 64 bit
2. Supported OS:
· Vista, 2008, windows 7
3. Remote power shell:
· Does not require exchange binaries to be installed on the admin machine which means you can also use 32 bit clients to remotely manage your exchange environment

B. EMC: Exchange Management Console:
1. Features:
· Built on remote PS and RBAC (Role based access control)
· Allows for multiple forest
· Cross premise 2010 management – including mailbox moves across organizations
· Recipient bulk edit – you can change mailbox attributes and permissions in bulk
· Power shell command logging

2. Demo: EMC (Exchange Management Console)
· You can see a extra node from EMC on the top for forest
· You can see all licensing info form forest level
· You can also view stuff for High availability; DAG
· You can select multiple mailbox and edit AD attributes and other exchange configurations
· The command log also even shows you the commands launched by EMC to view the information

C. ECP: Exchange control panel
1. What is it:
· Brower based management client for end users, administrator and specialist
· Accessible directly via url, owe, and outlook 2010
2. Who uses it:
· Admins and specialist: admins can delegate to specialist elg. Help desk operators, department admins and eDiscovery admins
· End users: comprehensive self service tools for end users
· Hosted customers: tenant administrators and tenant end users
3. Demo:
· Logging in as a regular user into owa. The user has restricted access so some of the buttons and dropped down menu is missing
· The user is able to change their own information
· Running delivery report: the users can populate the fields and run a search
· Managing distribution list (public groups): the user can created a new group and set permissions. You can set it so that other people can join by your approval.
· Logging in as another regular user
4. ECP architecture overview:
· Ajax based
· Shares some code with owa but two separate applications
Deployed on CAS
· ECP--> ASP.Net-->RBAC-->power shell
· Authentication: windows integrated, basic forms based
5. RBAC, how it changes UI:
· From a user POV, the options are not even there
D. RBAC (Role Based Access Control) in exchange 2010
1. Intro:
· RBAC has replaced permissions model in 2007: 2007 still used ACL
· Your role is defined by what you do
· Defined precise or broad roles and assignments based on the tasks that need to be performed. Its tied to power shell commands
· Includes self administration
· Used by EMC, EMS and ECP
2. Who can do what and where?
· Admins and end users
· Concept of roleGroup/USG
· You assign roles to people
3. What? Tasks that users can do. This come to be set of commandlets tied to following management roles:
· Organization management
· View only org management
· Recipientmangemente
· Um management
· Discovery management
4. Where?
· What is the scope? Could be recipient organization unit
· You can define by OU or by AD attribute
· You can also narrow by server scope
· You can further narrow permissions by use of Exclusive scopes. These are like exception clause.
5. custom management roles:
a. custom roles can be added to suit specific delegation requirements:
· roles are hierarchical, with build in role at the top
· role entries can only be removed from a role
b. steps to delegate a role:
· create the management role
· change the new role’s management role entries by removing role entries
· create a management scope
· assign the new management role

6. Demo:
· Get-managementroleassignment cmdlet: you can view all the role assignment
· Assigning helpdesk role: new-mangementroleassingment helpdesk –role mailboxadmin – user ‘contoso\jills’
· From ECP view you can see that she has my organization view: she can see mailboxes, can edit properties of mailbox, can create a new mailbox,
· Customizing the mailbox admin roles using power shell: set-mangementroleentry mailboxadmin\set-user –parameters department –removeparameter. This should make it read only
· Remove-mangaementroleEntry mailboxadmin\*new-mailbox*, you get a warning, continue this will remove permissions for mailbox management all together
· You don’t see the new button on the ECP, you can’t set the department field on the mailbox
· You can granular control over the set roles
7. RBAC role delegation:
a. Role membership is not a right to delegate
b. Role assignment delegation:
· Special kind of role assignment
· Delegation does not grant role permissions
c. RoleGroup delegation:
· Controlled through role group ownership
· Managed by parameter similar to DGs
· Ownership does not grant role group permissions
8. RBAC permission reporting
a. Get-managemtroleassignment:
· Effective roles for a user
· Effective user by role/scope/group
· Effective permissions to a writable object
E. Remote power shell:
1. New management architecture for power shell in 2010
a. Allows role based access control model:
· Restricted possessions allows rbac to hide completes and parameters
b. Role membership is not a right to delegate
· Rote power shell is always used to connect remotely to local host
· Enabled firewall and cross forest scenarios
c. No binaries scenarios:
· Exchange-cmdlet management from a client machine which does not have a exchange management tolls (exchange binaries) installed

2. How does it work?
a. Start off with 32 bit client
· Makes connection to IIS on the exchange server
· Wsman + rbac stack authorization, querying AD
· Role assignment given: you have a list of commandlets that you can run
· A PS session is created on the server with a list of commandlets that you can run
· You still get tab completion and all that good stuff
· After a command runs, it pipe lines on the client
3. How do I use it?
a. The beta way:
· Uses SSL
b. The rtm way:
· Uses http since authentication is via Kerberos
4. Demo:
· Running PS session from 32 bit client
· $rs = new-PSSsession –configurationName microsoft.exchhange –connectionURi http://slc-exch01.contoso.com/powershell/ -credential $cred
· Import-0pSSession$rs – this will go out to sever and fetch all commandlets available
· You can now run exchange commandlets

F. Monitoring:
1. Monitoring and reporting based on operations manager 2007
a. Supports 2007 sp1 or 2007 r2
b. Mp releasing concurrently with exchange 2010
2. Greatly reduced alert “noise:
a. Correlation engine
· Uses operations manager health model to hide symptoms alerts and leave root cause alerts for a faster problem resolution, fewer headaches
· Smarter alerts: exchange e2010 diagnostics specifically desinged for monitoring : scale ready, no more ;magic number threshold tuning
b. Better reporting
Exchange Server 2007 Service Pack 2 available in Q3 2009

Tuesday, May 12, 2009

Information Protection and Control in Microsoft Exchange Server 2010 - UNC314

A. Introduction:
1. Common practice for protecting information on email is usually done with disclaimer
2. 80% of all leaked information is due to accident


B. Exchange 2010 automatic protection via transport rule


C. Transport rules are created via templates (also done using power shell) where conditions such as sender, recipient, AD attribute, and key words can be used to define how you will protect your message. Message protection can restrict end recipient to only view the message and not print it, forward it, copy it... Exchange 2010 integrates Rights Management Service (RMS) to protect message and attachment.


D. RMS:
1. Allows for persistent protection
2. Using the RMS management console, you can restrict access and usage content


E. Demo: creation of transport rule using wizard and testing the rule
1. Simple rule was created for specified sender and for a pattern of text; xxx-xx-xxxx
2. Sender sends a message with matching text pattern
3. Recipient receives the message but it’s protected and you also see the tag ‘ DO NOT Forward...’. The option to forward is grayed out, recipient is able to view the message but can’t print, copy or save.


F. New Transport features:
1. Moderation – enables manager to review message for approval before going out to destined recipient. This is done via transport rules. 2010 adds usage of AD attributes into the mix of conditions and you can setup moderation between asst and agent or any set of users. you can also use text pattern on the message as a condition. This might be useful for messages going out to ‘all assistants’ or any other large distribution lists when sent by assistants. HR can moderate all message going to ‘all’ users groups before having it reach the masses. This is done via “arbitration mailbox”
2. 2010 transport rules can scan attachments. There is a built in OCR functionality and rules can set for text patterns in the body of message as well as attachments.
3. Current transport rule architecture does not allow for 3rd party plug in.


G. How does Information protection impact journaling? On a functionality level it doesn’t. when a message is journaled there are 2 versions of the message, the protected version and a unprotected version. So far as passing journal message to 3rd party archiving solution it’s seamless


H. How does message protection impact performing E-Discovery searches and archiving? It doesn’t all messages in archive stays encrypted. The process of E-Discovery will include decryption of message. All these features are available from OWA as well as full outlook version
I. Email Protection is only available on 2010, will not work on 2007.
1. Recipients not on exchange 2010 may have problems with RMS
2. Blackberry and RMS: there will be a 3rd party add on for blackberry for RMS

Exchange 2010 Enterprise Architecture – UNC316

A. key highlights:
1. still the same 5 role
2. all clients OWA/Outlook connect only to CAS server now
3. CAS server is the single end point for mailbox clients:
a. CAS removes DSPROXY by providing address service: CAS Array
b. RPC client access service :( only mailboxes on 2010 will follow this)
c. simplifies AD topology requirement for outlook
d. supports more concurrent connection to outlook

4. failover switch level on database level not servers level
5. 30 sec failover instead of 2-3 min as experienced on exchange 2007

B. Outlook anywhere and CAS:
1. In 2007 Two connections established: rpc in data and rpc out data. Having two sessions open with integration of SSL-ID was an issue. This would require two id’s. This created problems for profiles and killing sessions.

2. Use of the CAS array on 2010 fixes the two connection issue since CAS server does the dsproxy.
3. CAS servers are now the client (outlook, owa, active sync) end points providing not only RPC requests but also directory lookup
4. 2010 CAS servers will require more beefy hardware requirement due to additional roles

C. Writing to the directory: i.e., users updating a DL membership
1. CAS array now has nspi end point. Users make requests/changes. The change is pushed to GAL on backend via power shell

D. Better Scaling of mailbox connection:
1. windows 2008, allows for reuse of source ports for outbound CAS connection
2. Store process limited to 60k rpc connection on 2007 ~15k mailbox connection. With 2010, RPC end point/NSPI endpoint being on CAS:
a. RPC connection pools on CAS, 100k
b. RPC connection can be reutilized on CAS
c. recommended ISA:CAS ration 3:1.(for100% outlook anywhere) limitation here is ISA (32 bit) limited TCP connection per IS

E. Architectural consideration:
1. mixed mode deployment: as long as you have 2007 mailbox, you will need 2007 CAS server. users on 2007 mailboxes will utilize 2007 CAS and Hub servers for connection and transport where as 2010 users will only use 2010 HUB and CAS servers for delivery and connection. Delivery between version be handled by hub servers of both servers.
2. load balancing: for more than 8 CAS servers, consider Hardware load balancing

F. Exchange Transport: 2010 still uses JET but the following changes collectively results in reduced IO requirement
1. 2010 still uses JET.
2. ESE Page size went from 8K (2007) to 32K
3. ESE database page compression
4. Intrinsic long value record storage: on 2007 a typical message would require 13 IO. With Intrinsic long value record storage less page file is required per message
5. Increase in DB cache size to 1 GB
6. Check point depth increases from 128Mb to 512Mb
7. ESE version store maintenance

G. Transport server: shadow redundancy
1. Transport server keeps a copy of message locally while in transit.
a. Xshadow: hub retains copy in xshadow
b. xqdiscard: hub queries edge and once confirmed discards a local copy. By default every 5 min connection is made to edge to confirm delivery
c. once 3 request has been made and no response, hub sends message to edge2. What about duplicates? No guarantee that there is a duplicate detection for a non exchange external mail host but hotmail and gmail are able to detect

H. dumpster changes
1. db replication feedback is now used to control which messages remains in dumpster
2. when message has been relocated to all db copies message is truncated from dumpster
3. dumpster size is based on replication

I. Edge Server:
1. Better ADAM sync: with 2007 Edge syncs involved full rewrite of ADAM. With 2010 only deltas are synced since the last synchronization using cookie hash algorithm
2. Better SPAM filter: with 2007 only safe sender list was synchronized from client to edge. With 2010 safe senders, block list, and safe recipients are synchronized as well. 2010 also gives configurable safe list Quota and automatic safe sender update.

J. Mailbox Server:
1. Clients no longer make RPC connections to mailbox server
2. Store Schema changes allows for 100k items in mailbox from 20k (2007)
3. 30 sec failover: exchange no longer fails over entire exchange server. the new exchange is more database centric so failover is now a exchange task that involves database.
4. Public folders: still there in 2010, does not leverage CCR

K. UM:
1. Still recommend using physical stand alone server
2. Message preview with speech to text: this feature brings on additional overhead. Recommended memory is 8GB
3. Message preview is part of the message body so this can be searched against on a mailbox level and also across many mailboxes (E-Discovery). We should be able to search within EX as

L. Other improvements:
1. 2010 gives ability to measure message delivery and latency. You can now locate bottle neck in the message path and measure against set SLA
2. ISA will be split into 2 products: unfired access gateway and threat management gateway – both will be 64bit

keynote session

*office 2010 - technical preview available by 7/2009, by invitation only
*windows 7/windows 2008 R2
-Direct access to corporate without VPNning in - made possible by windows 2008 R2 setup/IPSec setup
-perform offline patching on VHD
-run scans on VHDs
-manage VHD from disk management mmc: you can attach the VHD and browse at a file level; you can also create a VHD from disk
Management,
-directly boot into VHD
-ship date for win 2008 R2 is towards end of the year

*windows 2008 R2
-x64 bit only
-ability to install just the server core and install only components needed
-can be managed remotely
-power shell cmdlets for managing AD objects and group policy
-compatibility mode: allows for use of older Hardware platform for hyper V.
-file classification infrastructure: addition of confidentiality tag, OCR capability so text in image format can also be
Searched
-tools for moving files to alternate locations, leaving shortcuts in the source

*Exchange 2010-OWA runs on Firefox and Mozilla

Monday, May 11, 2009

What's new in Mobile Messaging (WMB201)

some cool enhancements:

*conversation view and conversation action to deal with massive email*reply/foward information now from mobile

*if anyone has sent you email or they are on any of the CC, to, or from field it gets cached to a local device. when you start typing in the email address, you will start going trough this cache and of course the GAL and the contacts as well

*VM: the play button is available from the message. you can also view transcription. so far English and German is covered.
other languages will be supported

*get free busy information: you can go to contacts and see free/busy for the contact

*SMS Sync: text messages can be synced and captured to mailbox server
-you can send from OWA/Outlook 2010, if you go above 150 characters, it will start another message so you can continue to type. when sending form outlook, you are sending via phone.mailbox syncs with phone and phone sends out message. you don't need a 3d party app, the exchange servers sends 'relays' text message out via phone.
-if you get a new phone, it will sync the last 3 days of text messages-when you are sending SMS text form

*other improvemnt:-350% battery life improvement in communication mobile
-configure call forwarding
-java client for communicatior mobile
-EAS sync state upgrade
-POP/IMAP service discovery
-single number reach
-downloadable update CAB, Over the air (OTA). installs on WM 6.1+, auto detect at exchangne 2010, delivered OTA. if you have a registered device and you have 6.1, you will get a email with update link.

Intro to Exchange 2010

exchange 2010 Introduction: some cool improvements
*rights management template - built in wizard to create rights management policies around attachments as a tranport rule, you can restrict what files users share with other users inside and outside

*RBP - Role based administration-assign roles to users/IT for various exchange access; ie, compliance officer - legal user who can run searches across organization, human resource - update user information,...
-you can assign multiple roles to users. once the roles are assinged users can see these roles on a drop down menu

*ECP - exchange control panel (use to be owa options)-users can create and manage Distribution List-users can perform message tracking-users can update their own employee information

*you can move user mailbox without impacting email access/service to users!
Deployment flexibility:-70% reduction on IOPS over 2007-smoother IO patterns-resillience against corruption - especially with new ablity to have multiple nodes

*Anywhere access:-converation view/ ignore conversation view-scroll bar on OWA!!-conversation view: shows thread in continuous stream,-filtering option, you can enter any filter and not have to use sort by various fields-you can IM from OWAyou can run exchange on sata disks! with multiple nodes, deployed you can even go raidless!

*Email archiving-as a pier to primary mailbox-you can setup pol to move mail between mailbox to archive-can apply policy to any folder level - can be accessed by users to apply to their folderes-can apply to transport rules

*rights management at hub transport layers-support for both encrypted and non encrypted to allow for search

Thursday, May 7, 2009

routing from 2007 to 2003

we have 2 routing group connector in our mixed mode environment: a default one and a second one that I 've manually created. for some odd reason the default one started to build up queue and not deliver to the 2003 mailboxes. I saw an error '451 5.7.3 Cannot achieve Exchange Server authentication...' on the 2007 side. Exchange 2007 uses Certs/TLS while 2003 doens't so integrated wind0ws authentication is used to allow communication. I checked the permission setting on the access tab of the 2003 virtual smtp and it was so. for some reason la-exchhub02 couldn't connect to bh-evs01 where as other hub servers were doing this. the thing was hub transport is that if one is designate as source transport server, the other hub servers should be able deliver to 2003 mailbox servers by passing their mails to the designagted hub server. this was happening but some emails were just getting stuck on the routing group. this was resolved by editing the 'interop rgc's source transport server on the 2007 side. by using 'Set-RoutingGroupConnector -Identity "f958a918-6797-41af-a8ad-5fddb6225063" -SourceTransportServers "la-exchhub01","la-exchhub02"'

after this restarting the transport service cleared the queue

understanding routing groups

http://itprosecure.com/blogs/exchange_2007_administration/archive/2009/01/19/exchange-2007-understanding-the-routing-group-connector-between-an-exchange-2003-and-exchange-2007-messaging-environment.aspx

Tony's Dump Site

how to create new routing groups:

New-RoutingGroupConnector -name "" -SourceTransportServers "" -TargetTarnsportServers "" -cost100 -bidirectional $true -PublicFolderReferralsEnabled $true New-RoutingGroupConnector -name "" -SourceTransportServers "" -TargetTarnsportServers "" -cost100 -bidirectional $true -PublicFolderReferralsEnabled $true

Tony Lee's Blog

creating a anonymous relay involes a two step. the first step is support via EMC but the second part must be done using powershell.

Get-ReceiveConnector "CRM Application" Add-ADPermission -User "NT AUTHORITY\ANONYMOUS LOGON" -ExtendedRights "ms-Exch-SMTP-Accept-Any-Recipient

steps to first part is found here: http://technet.microsoft.com/en-us/library/bb232021.aspx

Tony Lee's Blog

following these steps resolved the issue:

1. Click Start, click All Programs, click Microsoft Exchange Server 2007 and click Exchange Management Shell.
2. Run the following cmdlet to view the current setting of the default domain value on your receive connector:Get-ReceiveConnector -identity flIn the output, notice that the value of DefaultDomain is blank by default.
3. Run the following cmdlet to set the default domain value:Set-ReceiveConnector -identity “” -DefaultDomain “”For example, Set-ReceiveConnector -identity “Default EX2007MAIL” -DefaultDomain “contoso.com”
4. Run Get-ReceiveConnector -identity fl cmdlet again and verify that the DefaultDomain has been set in the output

What is BlogThis! ? - Blogger Help

little problem with 2007 exchange hub connector this morning. the 2007 receiver connector by default does not include the sending domain. this forces all senders to populate internal recipient address in name@domain.com format where as in 2003 it automatically set.

http://support.microsoft.com/kb/944302
Testing from phone