Thursday, 24 March 2022

Azure SQL

 

  • A fully managed database built upon the SQL Server engine.
  • SLA durability up to 99.995%.
  • SQL Databases Resource type:
    • Single Database – offers serverless and hyperscale storage (up to 100TB).
    • Elastic Pool – a collection of databases with a shared set of resources.
    • Database Server – manage groups of single databases and elastic pools.
  • SQL Managed Instances are for migrations “lift-and-shift” to the cloud.
    • The features of both SQL Server database engine and Azure SQL are available in SQL Managed Instance. 
      • PaaS benefits – Azure will handle all infrastructure management.
      • Business continuity – the data are protected with automated backups.
      • Security and compliance – supports native VNet implementation and it is exposed only through a private IP address.
      • Management operations –  automatically deploy new instances, update instance properties, and delete instances that are no longer needed.
    • Automate the migration of existing SQL Server instance to SQL Managed Instance with Azure Data Migration Service
    • Azure Hybrid Benefit for SQL Server allows you to exchange existing licenses to get discounted rates on SQL Managed Instance.
  • SQL Virtual Machines are used for applications requiring OS-level access.
  • Endpoint: <server_name>.database.windows.net
  • vCore-based service tiers:
    • General Purpose is for common workloads.
    • Hyperscale is appropriate for online transaction processing (OLTP) and hybrid transactional analytical workloads (HTAP).
    • Business Critical is best for OLTP applications with high transaction rates and low IO latency.
  • Azure Hybrid Benefit for SQL Server enables you to use your SQL Server licenses to pay a reduced rate on Azure SQL.
  • Azure Data Studio is a modern cross-platform database tool with customizable code snippets, lightning-fast IntelliSense, useful peek definitions, and an integrated terminal to run other SQL tools.

Monitoring

  • You can use Intelligent Insights to continuously monitor your Azure SQL usage and detect disruptive events that may lead to poor database performance.
  • Azure SQL Analytics can be used to monitor your databases across multiple subscriptions. It can collect and visualize key performance metrics of your databases and enables you to create custom monitoring rules and alerts.
  • Automatic tuning in Azure SQL continuously monitors queries executed on your database, and automatically improves the performance using artificial intelligence. 

Networking

  • Private endpoint connections provide access to all databases in the server.
  • Allow communications from all resources inside the Azure boundary with firewall rules.

Security

  • You can use Advanced Data Security (ADS) for data classification, vulnerability assessment, and advanced threat protection.

Pricing

  • The resources are billed hourly at a fixed rate based on the service tier and compute size you choose.
  • You are billed for outgoing Internet traffic.

Azure Cosmos DB

 

  • Globally distributed database that supports NoSQL.
  • A fully-managed database service with turnkey global distribution and transparent multi-master replication.

Features

  • Cosmos DB offers encryption at rest.
  • It replicates every partition across all the regions.
  • CosmosDB offers single-digit millisecond reads and writes in all regions.
  • Supports automatic failover during a regional outage.
  • Consistency Levels: Strong, Bounded Staleness, Session, Consistent Prefix, and Eventual
  • You can set either standard (manual) or autoscale provisioned throughput on your databases and containers.
  • Monitor both the provisioned autoscale max RU/s and the current throughput (RU/s) of the system with Azure Monitor metrics.

Consistency Levels

  • Strong – reads are guaranteed to return the most recent committed version of an item.
  • Bounded Staleness – is for low write latencies but requires a total global order guarantee.
  • Session – reads are guaranteed to honor the consistent-prefix, monotonic reads, monotonic writes, read-your-writes, and write-follows-reads guarantees.
  • Consistent Prefix – updates that are returned will contain some prefix of all the updates.
  • Eventual – has no ordering guarantee for reads.

Security

  • Encryption at rest is applied automatically.
  • Uses AES-256 encryption in all regions.
  • You can use the keys that are managed by Microsoft or customer-managed keys.
  • Two types of keys to authenticate users:
    • Master Keys for administrative resources.
    • Resources Tokens for application resources.
  • Detect suspicious activities which indicate unusual and potentially harmful attempts to access or exploit databases with Advanced Threat Protection.

Pricing

  • Elastically scale the provisioned throughput and storage for your databases according to your needs and only pay for the throughput and storage needed.
  • Cosmos DB throughput per second and request unit consumption varies by operation and JSON document.
  • Additional backup copies will be billed as total GBs of data stored (first two copies are free).
  • You provision the number of RUs for your application on a per-second basis in increments of 100 RUs per second. You are billed on an hourly basis.
  • With autoscale for containers, you pay per hour for the highest RU/s that the system scaled up to within the hour.

Azure Archive Storage

 

  • Store rarely accessed data which are held for a period of 180 days.
  • Snapshots are not applicable to archive storage.

Features

  • It supports 2 rehydrate priorities: High and Standard
    • Standard (Default) – rehydration request may take up to 15 hours.
    • High – rehydration request may finish in under 1 hour for objects under 10 GB in size.
  • Long-term backup, secondary backup, and archival datasets
  • Lowest storage costs but the highest data rehydrate and access costs
  • To read data in archive storage, you need to change the blob tier to hot or cold first.
  • Compliance and archival data that must be preserved and are hardly ever accessed.
  • Archive storage only supports block blobs
  • If a blob is in the archive tier, it can’t be overwritten, unlike in hot or cool tier
  • Archive storage cannot be set as a default account access tier
  • Archive storage is initially available in selected regions.
  • Blob index tags can be read, set, or modified while in the archive.
  • You can only copy archive blobs within the same storage account.
  • Encrypted data transfer to the cloud using HTTPS, and using 256-bit AES keys to automatically protect the data at rest.

Use Cases

  • It is mainly used in long-term backup retention
  • If you need to minimize your cost, use Archive Storage to create a low cost, content archiving solution.
  • Archive storage provides secure, globally compliant storage for sensitive data.
  • You can also use Archive storage if you have a large amount of data that needs to be preserved.

Pricing

  • Blobs are stored for at least 180 days in the archive tier. Deleting or rehydrating archived blobs before the minimum number of days will incur early deletion fees.
  • Charges on data access increases as the tier gets cooler. For data in the cool and archive access tier, you’re charged a per-gigabyte data access charge for reads.

Azure Table Storage

 

  • A NoSQL key-value store for large semi-structured datasets.
  • Supports flexible data schema.
  • Performs OData-based queries

Features

  • Allows you to store and query huge sets of structured, non-relational data. And as demand grows, your tables will scale-out.
  • Scale-up without having to manually shard your dataset.
  • The data is replicated three times within a region using geo-redundant storage.
  • An entity has a limit of 1MB in size.
  • Store data sets that do not require complex joins, foreign keys, or stored procedures, and can be denormalized for fast access.
  • Table storage is used to store flexible data sets such as user data for web applications, device information, or other types of metadata the service requires.
  • You can store any number of entities in a table, up to the storage account’s capacity limit.

Azure Queue Storage

 

  • Durable queues for large-volume cloud services.
  • Store large numbers of messages.
  • Queue messages may have a size of up to 64 KB.

azure queue storage

Features

  • Asynchronous message queueing to communicate between components of the application.
  • Built to be scalable and withstand the failure of individual components
  • Monitor the length of the queue to add elasticity to your application, and hibernate or deploy additional nodes depending on customer demand.

Pricing

  • No upfront cost and termination fees.
  • You only pay for what you use.

Azure Files

 

  • Offers fully managed cloud-based file storage that can be accessed through the industry-standard server message block (SMB) protocol.

Features

  • Mount your Azure File share from Windows, Linux, or macOS.
  • Azure File Sync enables you to access your data from SMB, REST, or even on-premises.
  • Encrypt data at rest and in transit using SMB 3.0 and HTTPS.
  • Lift and shift applications to the cloud, where the application data is moved to Azure Files, and the application continues to run on-premises.
  • Store configuration files in a centralized location where they can be accessed from many application instances.
  • Azure Files provides the capability of taking share snapshots of file shares.

Storage Tiers

  • Premium file shares (SSD)
    • High performance & low latency, within single-digit milliseconds for most IO operations.
    • For IO-intensive workloads.
  • Standard file shares (HDD)
    • Reliable performance for IO workloads which are less latency-sensitive.
  • If you created either a premium or a standard file share, you cannot automatically convert it to the other tier.

Detail

Premium

Standard

Billing model

Provisioned Billing Model, pay for how much storage you provision rather than how much storage you actually ask for.

Pay-As-You-Go Model, the bill will increase if you use (read/write/mount) the Azure file share more.

Redundancy options

It is available for locally redundant (LRS) and zone redundant (ZRS) storage.

It is available for locally redundant, zone redundant, geo-redundant (GRS), and geo-zone redundant (GZRS) storage.

Maximum size of file share

Provisioned for up to 100 TiB.

5 TiB by default, 100 TiB for locally redundant or zone redundant storage accounts.

Regional availability

File shares are not available in each region, but zone redundant support is available in a smaller subset of regions.

Available in every Azure region.

 

Supported Devices

  • To use an Azure file share outside of the Azure region the OS must support SMB 3.0
  • To mount an Azure file sharing on Windows, you must have access to port 445.

Windows version

SMB version

Mountable in Azure VM

Mountable on-premises

Windows Server 2019

SMB 3.0

Yes

Yes

Windows 101

SMB 3.0

Yes

Yes

Windows Server semi-annual channel2

SMB 3.0

Yes

Yes

Windows Server 2016

SMB 3.0

Yes

Yes

Windows 8.1

SMB 3.0

Yes

Yes

Windows Server 2012 R2

SMB 3.0

Yes

Yes

Windows Server 2012

SMB 3.0

Yes

Yes

Windows 73

SMB 2.1

Yes

No

Windows Server 2008 R23

SMB 2.1

Yes

No

 

  • Linux clients can also access the file storage through the SMB protocol.

Linux distribution

SMB 2.1 (Mounts on VMs within same Azure region)

SMB 3.0 (Mounts from on-premises and cross-region)

Ubuntu

14.04+

16.04+

Red Hat Enterprise Linux (RHEL)

7+

7.5+

CentOS

7+

7.5+

Debian

8+

10+

openSUSE

13.2+

42.3+

SUSE Linux Enterprise Server

12+

12 SP3+

 

Encryption

  • By default, encrypted with Microsoft-managed keys and responsible for rotating them on a regular basis.
  • Using Microsoft-managed keys, you can also choose to manage your own keys, which gives you control over the rotation process.
  • With customer-managed keys, Azure file storage is authorized to access your keys to fulfill read and write requests from your clients.

Networking

  • SMB uses port 445.
  • Accessible from anywhere, via the public endpoint of the storage account.
  • Azure file shares over an ExpressRoute or VPN connection:
    • Tunneling into a virtual network, even if port 445 is blocked.
    • Private endpoints give you a dedicated IP address from within the address space of the virtual network.
    • Allows you to configure DNS forwarding.
  • The UNC path format of an Azure File Share is: \\<storageAccountName>.file.core.windows.net\

Azure File Sync

  • Transform an on-premises (or cloud) Windows Server into a quick cache of your Azure file share.
  • Use Azure File Sync agent to synchronize files from a server to an Azure file share. 
  • To create sync groups, you need to deploy a Storage Sync Service.
  • A sync group defines the sync relationship between a cloud endpoint and a server endpoint.
    • Cloud endpoint – represents an Azure file share and multiple server endpoints. 
    • Server endpoint – a path registered on the Windows Server.
    • When you make changes to your cloud endpoint or server endpoint, your files are automatically synced to your sync group’s remaining endpoints.
    • When you make a change directly to the cloud endpoint, Azure files must first detect it via a change detection job, which only happens once every 24 hours.
    • A change detection job enumerates all the files in the file share and compares it to the sync version of that file. When the change detection job determines that there are changes, Azure File sync will initiate a sync session.
  • The sync group you created should only have one cloud endpoint.
  • A sync group may have server endpoints with different Active Directory memberships, even if they are not domain-joined.
  • The storage accounts used for Azure Files deployments are:
    • General purpose version 2 (GPv2) storage accounts
    • FileStorage storage accounts
  • You can use cloud tiering to cache frequently accessed files locally on the server.
  • Only NTFS volumes are supported; ReFS, FAT, FAT32, and other file systems are not supported.
  • The service supports interop with DFS Namespaces (DFS-N) and DFS Replication (DFS-R).
    • DFS-N allows you to group shared folders located on multiple servers into one or more logically structured namespaces.
    • DFS-R enables you to replicate folders across multiple servers and sites.
  • Azure File Sync has three layers of encryption:
    • Encryption at rest (Windows Server)
    • Encryption in transit
    • Encryption at rest (Azure file share)

Azure Disk Storage

 

  • Block-level storage volumes for Azure Virtual Machines
  • Disk Storage is a managed disk that is designed for 99.999% availability.
  • You can create 50,000 VM disks for each region.

Features

  • Different types of storage options: Standard HDD, Standard SSD, Premium SSD, and Ultra Disk volumes up to 64 TiB.
  • The OS disk has a maximum capacity of 4,095 GiB.
  • The maximum size of the data disk is 32,767 GiB.
  • Managed disks are integrated with the availability sets to ensure that the VM disks are separated from each other in an availability set to prevent a single failure point.
  • You can assign specific permissions for a managed disk to one or more users using Azure RBAC.
  • You can use the temporary disk to store data such as page or swap files
  • Ephemeral OS disks for stateless applications
  • Attach a managed disk to multiple virtual machines (VMs) simultaneously using Azure shared disks.
  • With snapshots, you can take a back up of your managed disks at any given point in time.

Disk Types

  • Standard HDD
    • Low cost and suitable for backups.
    • Write latencies under 10ms.
    • Read latencies under 20ms.
  • Standard SSD
    • Consistent performance at lower IOPS levels.
    • Higher reliability, scalability, and lower latency over HDD.
  • Premium SSD
    • High-performance and low-latency disk for VMs.
    • Consistent IOPS, and throughput.
    • Offers disk bursting and can burst their IOPS per disk up to 3,500 and their bandwidth up to 170 Mbps.
    • Peak burst limit of 30 mins
  • Ultra Disk
    • High throughput, high IOPS, and consistent low latency disk storage
    • Only supports un-cached reads and un-cached writes
    • Doesn’t support disk snapshots, VM images, availability sets, Azure Dedicated Hosts, or Azure disk encryption.
    • The integration with Azure Backup or Azure Site Recovery is not supported.

Detail

Standard HDD

Standard SSD

Premium SSD

Ultra Disk

Disk type

HDD

SSD

SSD

SSD

Scenario

Backup, non-critical, infrequent access

Web servers, and light applications of enterprise

Production and performance sensitive workloads

IO-intensive workloads, top tier databases, and other transaction-heavy workloads

Max disk size

32,767 GiB

32,767 GiB

32,767 GiB

65,536 GiB

Max throughput

500 MB/s

750 MB/s

900 MB/s

2,000 MB/s

Max IOPS

2,000

6,000

20,000

160,000

 

Encryption

  • Server-Side Encryption (SSE) is performed by the storage service.
  • Azure Disk Encryption (ADE) can be enabled on the OS and data disks.
  • Encrypted using 256-bit AES encryption.
  • For standard HDDs, standard SSDs, and premium SSDs: disabling or deleting the key will automatically shut down all the VMs with disks using that key.
  • If you disable or delete a key, any virtual machines with ultra disks using the key won’t automatically shut down.
  • Once you enable end-to-end encryption, temp disks and ephemeral OS disks are encrypted with platform-managed keys.

Pricing

  • Managed disk size is billed on the provisioned size.
  • Snapshots are charged based on the size used.
  • Outbound data transfers incur billing for bandwidth usage.
  • You are charged for the number of transactions that you perform on a managed disk (the number of read and write data operations performed).