eygle.com   eygle.com
eygle.com eygle
eygle.com  
 
Digest Net: March 2011 Archives

March 2011 Archives

利用RMAN进行基于时间点的修复,有时候需要设置nls_date_format参数,才能正确执行,如:

C:>set NLS_DATE_FORMAT=YYYY-MM-DD HH24:MI:SS
C:>rman nocatalog target /

RMAN>run {
allocate channel c1 type disk;
alloctae channel c2 type disk;
set until time='2011-04-21:11:09:01';
restore database;
recover database;
sql 'alter database open resetlogs';
release channel c1;
release channel c2;
}

在RMAN中修改日期格式可以使用如下方式:
RMAN>run { sql "alter session set nls_date_format=''yyyy-mm-dd hh24:mi:ss''";}

在SQL*Plus中可以使用如下方式:

SQL>alter session set nls_date_format='yyyy-mm-dd hh24:mi:ss';
SQL>select sysdate from dual;

SYSDATE
-------------------
2011-04-21 11:11:20


如何高效处理全球信息一直是Google的狂热追求,而其自行开发的C++编程工具MapReduce就在其中扮演着关键角色。它可以多线程同时执行大规模数据集(1TB以上)的并行运算,是日常超大计算量的完美方案。

Google今天自豪地宣布,他们只需要短短68秒钟就能完成对1TB数据的排序处理。这些数据都是未压缩的文本文件,使用Google File System文件系统存储在1000台计算机上。在此之前,Google处理分布于910台计算机上的等量数据需要耗时209秒,效率只有现在的三分之一 左右。

当然,信息爆炸时代的数据量远远不只是TB级别,更常见的是一千倍的PB级别。在今年1月份的时候,Google MapReduce平均每天的数据处理量是20PB,相当于美国国会图书馆今年5月份存档网络数据的240倍。

那么Google MapReduce对4000台计算机上的1PB数据进行排序处理需要多长时间呢?答案是6小时零2分钟。放眼全球,除了Google还没有谁具备这种高速处理能力。

Google还透露,这1PB数据是存储在48000个硬盘上的(当然并没有全部填满),不过考虑到测试的持续时间、涉及的硬盘数量、硬盘的使用寿命,每次进行测试都会有至少一块硬盘挂掉。为此Google文件系统会为每个文件备份三个拷贝,并分别放在三块硬盘上。

Link:http://www.cioage.com/art/200901/77364.htm

At Google we are fanatical about organizing the world's information. As a result, we spend a lot of time finding better ways to sort information using MapReduce, a key component of our software infrastructure that allows us to run multiple processes simultaneously. MapReduce is a perfect solution for many of the computations we run daily, due in large part to its simplicity, applicability to a wide range of real-world computing tasks, and natural translation to highly scalable distributed implementations that harness the power of thousands of computers.

In our sorting experiments we have followed the rules of a standard terabyte (TB) sort benchmark. Standardized experiments help us understand and compare the benefits of various technologies and also add a competitive spirit. You can think of it as an Olympic event for computations. By pushing the boundaries of these types of programs, we learn about the limitations of current technologies as well as the lessons useful in designing next generation computing platforms. This, in turn, should help everyone have faster access to higher-quality information.

We are excited to announce we were able to sort 1TB (stored on the Google File System as 10 billion 100-byte records in uncompressed text files) on 1,000 computers in 68 seconds. By comparison, the previous 1TB sorting record is 209 seconds on 910 computers.

Sometimes you need to sort more than a terabyte, so we were curious to find out what happens when you sort more and gave one petabyte (PB) a try. One petabyte is a thousand terabytes, or, to put this amount in perspective, it is 12 times the amount of archived web data in the U.S. Library of Congress as of May 2008. In comparison, consider that the aggregate size of data processed by all instances of MapReduce at Google was on average 20PB per day in January 2008.

It took six hours and two minutes to sort 1PB (10 trillion 100-byte records) on 4,000 computers. We're not aware of any other sorting experiment at this scale and are obviously very excited to be able to process so much data so quickly.

An interesting question came up while running experiments at such a scale: Where do you put 1PB of sorted data? We were writing it to 48,000 hard drives (we did not use the full capacity of these disks, though), and every time we ran our sort, at least one of our disks managed to break (this is not surprising at all given the duration of the test, the number of disks involved, and the expected lifetime of hard disks). To make sure we kept our sorted petabyte safe, we asked the Google File System to write three copies of each file to three different disks.

Significantly improved handling of the so-called "stragglers" (parts of computation that run slower than expected) was a key software technique that helped sort 1PB. And of course, there are many other factors that contributed to the result. We'll be discussing all of this and more in an upcoming publication. And you can also check out the video from our recent Technology RoundTable Series.

Link:http://googleblog.blogspot.com/2008/11/sorting-1pb-with-mapreduce.html

海量数据 10 PB 到底有多大?

| 12 Comments
原文链接。原文时间:2011.1.16

1PB(PetaByte)等于2的50次方,等于1024TB,10PB是目前整个互联网现有数据存档的两倍。但这些数字听上去太抽象了,10PB到底有多大呢?数据存储服务公司Backblaze制作了几张图片,能让大家一目了然:

按硬盘盒的数量算,10PB数据可存满137个硬盘盒,每个盒子里装有45块硬盘。

按硬盘数算,10 PB相当于2280个2TB硬盘,3166个1.5TB硬盘,再加上749个1TB硬盘。

Backblaze现有硬盘数量为6195个,这些硬盘摞在一起高达2968英尺(约为905米),比迪拜的哈利法塔还高。


Oracle Innovators-Bill Bridge Conversations

Link http://www.oracle.com/innovation/innovator-bill-bridge.html

Bill Bridge 是 Oracle ASM技术的架构师,是他最早提出和发起了ASM架构,这是在Oracle网站上找到的对他的采访,目前该链接已经失效。

Automatic Storage Management
Automatic Storage Management (ASM) is a custom file system/volume manager for the Oracle database, built into the Oracle kernel. It provides the best practices for Oracle Database 10g and Oracle Database 11g storage by simplifying both the storage and the management of data contained in the Oracle database.

Architect
Location: Redwood City, California
Product:Oracle Automatic Storage Management

Q:Why is Automatic Storage Management innovative?

A:The data files of a database tend to be large files. With traditional volume managers, you can't figure out how your data is mapped to the physical spindle. You can wind up with one disk that gets way too much I/O and pulls the whole system down. ASM breaks data files into pieces that are big enough so that you get good sequential performance if you read the whole piece and then spread them across multiple disks. By spreading data evenly, ASM maintains even disk performance and gives customers better disk utilization, so you don't have hot disks. As a result, customers can manage more data with fewer resources. Most Oracle Real Application Clusters users use ASM.

Q:What's the most innovative product you've worked on--and why?

A:ASM. In 1996 I came up with the fundamental concept. The first production release was in 2003 for Oracle Database 10g Release 1.0, where it was one of the most significant new features. It's innovative because it was the first file system to evenly balance load across multiple disks and reorganize data online. By keeping a pointer to every good-sized piece of storage, you could relocate them one at a time and do it incrementally, while the system is up and running. Then you only have to move the data that you need to move, not all of it.

Q:How do you think Oracle's innovative culture benefits our customers?

A:It keeps us on the leading edge of database technology. We have the best databases in the world. And it keeps getting better. That benefits our customers because they can do more things with data more efficiently.

Q:What innovative technology are you excited about right now?

A:Virtual machines and the ability to scale data centers by having lots and lots of identical servers as well as virtual servers that can migrate between them. It's the data center of the future. It's going to require lots of innovation.

Q:Who in world history do you think of as the most innovative?

A:Leonardo da Vinci. How cool would it be to drive down the freeway with him in the passenger seat?

Q:What's the most innovative product--not Oracle's--that you know of?

A:A lot of innovation, I believe, is happening in biotech, like the sequencing of the human genome.


Enqueue Waits in Oracle Database 10g

转引自: http://oracle-dox.net/McGraw.Hill-Oracle.Wait.Interf/8174final/LiB0063.html

In Oracle Database 10g Release 1, each enqueue type is represented by its own wait event, making it much easier to understand exactly what type of enqueue the session is waiting for. You do not need to decipher the values from the P1, P2, P3, P1RAW, P2RAW, and P3RAW columns in the V$SESSION_WAIT or the V$SESSION view.

The following table lists all the enqueue waits in Oracle Database 10g Release 1 and describes what the enqueue is for. This information is available in the X$KSQST structure. The aggregated statistics for each of these enqueue types is displayed by the view V$ENQUEUE_STAT.

Enqueue Type

Description

enq: AD - allocate AU

Synchronizes accesses to a specific OSM (Oracle Software Manager) disk AU

enq: AD - deallocate AU

Synchronizes accesses to a specific OSM disk AU

enq: AF - task serialization

Serializes access to an advisor task

enq: AG - contention

Synchronizes generation use of a particular workspace

enq: AO - contention

Synchronizes access to objects and scalar variables

enq: AS - contention

Synchronizes new service activation

enq: AT - contention

Serializes alter tablespace operations

enq: AW - AW$ table lock

Allows global access synchronization to the AW$ table (analytical workplace tables used in OLAP option)

enq: AW - AW generation lock

Gives in-use generation state for a particular workspace

enq: AW - user access for AW

Synchronizes user accesses to a particular workspace

enq: AW - AW state lock

Row lock synchronization for the AW$ table

enq: BR - file shrink

Lock held to prevent file from decreasing in physical size during RMAN backup

enq: BR - proxy-copy

Lock held to allow cleanup from backup mode during an RMAN proxy-copy backup

enq: CF - contention

Synchronizes accesses to the controlfile

enq: CI - contention

Coordinates cross-instance function invocations

enq: CL - drop label

Synchronizes accesses to label cache when dropping a label

enq: CL - compare labels

Synchronizes accesses to label cache for label comparison

enq: CM - gate

Serializes access to instance enqueue

enq: CM - instance

Indicates OSM disk group is mounted

enq: CT - global space management

Lock held during change tracking space management operations that affect the entire change tracking file

enq: CT - state

Lock held while enabling or disabling change tracking to ensure that it is enabled or disabled by only one user at a time

enq: CT - state change gate 2

Lock held while enabling or disabling change tracking in RAC

enq: CT - reading

Lock held to ensure that change tracking data remains in existence until a reader is done with it

enq: CT - CTWR process start/stop

Lock held to ensure that only one CTWR (Change Tracking Writer, which tracks block changes and is initiated by the alter database enable block change tracking command) process is started in a single instance

enq: CT - state change gate 1

Lock held while enabling or disabling change tracking in RAC

enq: CT - change stream ownership

Lock held by one instance while change tracking is enabled to guarantee access to thread-specific resources

enq: CT - local space management

Lock held during change tracking space management operations that affect just the data for one thread

enq: CU - contention

Recovers cursors in case of death while compiling

enq: DB - contention

Synchronizes modification of database wide supplemental logging attributes

enq: DD - contention

Synchronizes local accesses to ASM (Automatic Storage Management) disk groups

enq: DF - contention

Enqueue held by foreground or DBWR when a datafile is brought online in RAC

enq: DG - contention

Synchronizes accesses to ASM disk groups

enq: DL - contention

Lock to prevent index DDL during direct load

enq: DM - contention

Enqueue held by foreground or DBWR to synchronize database mount/open with other operations

enq: DN - contention

Serializes group number generations

enq: DP - contention

Synchronizes access to LDAP parameters

enq: DR - contention

Serializes the active distributed recovery operation

enq: DS - contention

Prevents a database suspend during LMON reconfiguration

enq: DT - contention

Serializes changing the default temporary table space and user creation

enq: DV - contention

Synchronizes access to lower-version Diana (PL/SQL intermediate representation)

enq: DX - contention

Serializes tightly coupled distributed transaction branches

enq: FA - access file

Synchronizes accesses to open ASM files

enq: FB - contention

Ensures that only one process can format data blocks in auto segment space managed tablespaces

enq: FC - open an ACD thread

LGWR opens an ACD thread

enq: FC - recover an ACD thread

SMON recovers an ACD thread

enq: FD - Marker generation

Synchronization

enq: FD - Flashback coordinator

Synchronization

enq: FD - Tablespace flashback on/off

Synchronization

enq: FD - Flashback on/off

Synchronization

Enqueue Type

Description

enq: FG - serialize ACD relocate

Only 1 process in the cluster may do ACD relocation in a disk group

enq: FG - LGWR redo generation enq race

Resolves race condition to acquire Disk Group Redo Generation Enqueue

enq: FG - FG redo generation enq race

Resolves race condition to acquire Disk Group Redo Generation Enqueue

enq: FL - Flashback database log

Synchronizes access to Flashback database log

enq: FL - Flashback db command

Synchronizes Flashback Database and deletion of flashback logs

enq: FM - contention

Synchronizes access to global file mapping state

enq: FR - contention

Begins recovery of disk group

enq: FS - contention

Synchronizes recovery and file operations or synchronizes dictionary check

enq: FT - allow LGWR writes

Allows LGWR to generate redo in this thread

enq: FT - disable LGWR writes

Prevents LGWR from generating redo in this thread

enq: FU - contention

Serializes the capture of the DB feature, usage, and high watermark statistics

enq: HD - contention

Serializes accesses to ASM SGA data structures

enq: HP - contention

Synchronizes accesses to queue pages

enq: HQ - contention

Synchronizes the creation of new queue IDs

enq: HV - contention

Lock used to broker the high watermark during parallel inserts

enq: HW - contention

Lock used to broker the high watermark during parallel inserts

enq: IA - contention

Information not available

enq: ID - contention

Lock held to prevent other processes from performing controlfile transaction while NID is running

enq: IL - contention

Synchronizes accesses to internal label data structures

Enqueue Type

Description

enq: IM - contention for blr

Serializes block recovery for IMU txn

enq: IR - contention

Synchronizes instance recovery

enq: IR - contention2

Synchronizes parallel instance recovery and shutdown immediate

enq: IS - contention

Synchronizes instance state changes

enq: IT - contention

Synchronizes accesses to a temp object's metadata

enq: JD - contention

Synchronizes dates between job queue coordinator and slave processes

enq: JI - contention

Lock held during materialized view operations (such as refresh, alter) to prevent concurrent operations on the same materialized view

enq: JQ - contention

Lock to prevent multiple instances from running a single job

enq: JS - contention

Synchronizes accesses to the job cache

enq: JS - coord post lock

Lock for coordinator posting

enq: JS - global wdw lock

Lock acquired when doing wdw ddl

enq: JS - job chain evaluate lock

Lock when job chain evaluated for steps to create

enq: JS - q mem clnup lck

Lock obtained when cleaning up q memory

enq: JS - slave enq get lock2

Gets run info locks before slv objget

enq: JS - slave enq get lock1

Slave locks exec pre to sess strt

enq: JS - running job cnt lock3

Lock to set running job count epost

enq: JS - running job cnt lock2

Lock to set running job count epre

enq: JS - running job cnt lock

Lock to get running job count

enq: JS - coord rcv lock

Lock when coord receives msg

enq: JS - queue lock

Lock on internal scheduler queue

enq: JS - job run lock - synchronize

Lock to prevent job from running elsewhere

enq: JS - job recov lock

Lock to recover jobs running on crashed RAC inst

Enqueue Type

Description

enq: KK - context

Lock held by open redo thread, used by other instances to force a log switch

enq: KM - contention

Synchronizes various Resource Manager operations

enq: KP - contention

Synchronizes kupp process startup

enq: KT - contention

Synchronizes accesses to the current Resource Manager plan

enq: MD - contention

Lock held during materialized view log DDL statements

enq: MH - contention

Lock used for recovery when setting Mail Host for AQ e-mail notifications

enq: ML - contention

Lock used for recovery when setting Mail Port for AQ e-mail notifications

enq: MN - contention

Synchronizes updates to the LogMiner dictionary and prevents multiple instances from preparing the same LogMiner session

enq: MR - contention

Lock used to coordinate media recovery with other uses of datafiles

enq: MS - contention

Lock held during materialized view refresh to set up MV log

enq: MW - contention

Serializes the calibration of the manageability schedules with the Maintenance Window

enq: OC - contention

Synchronizes write accesses to the outline cache

enq: OL - contention

Synchronizes accesses to a particular outline name

enq: OQ - xsoqhiAlloc

Synchronizes access to olapi history allocation

enq: OQ - xsoqhiClose

Synchronizes access to olapi history closing

enq: OQ - xsoqhistrecb

Synchronizes access to olapi history globals

enq: OQ - xsoqhiFlush

Synchronizes access to olapi history flushing

enq: OQ - xsoq*histrecb

Synchronizes access to olapi history parameter CB

enq: PD - contention

Prevents others from updating the same property

enq: PE - contention

Synchronizes system parameter updates

Enqueue Type

Description

enq: PF - contention

Synchronizes accesses to the password file

enq: PG - contention

Synchronizes global system parameter updates

enq: PH - contention

Lock used for recovery when setting proxy for AQ HTTP notifications

enq: PI - contention

Communicates remote Parallel Execution Server Process creation status

enq: PL - contention

Coordinates plug-in operation of transportable tablespaces

enq: PR - contention

Synchronizes process startup

enq: PS - contention

Parallel Execution Server Process reservation and synchronization

enq: PT - contention

Synchronizes access to ASM PST metadata

enq: PV - syncstart

Synchronizes slave start_shutdown

enq: PV - syncshut

Synchronizes instance shutdown_slvstart

enq: PW - prewarm status in dbw0

DBWR0 holds this enqueue indicating pre-warmed buffers present in cache

enq: PW - flush prewarm buffers

Direct Load needs to flush prewarmed buffers if DBWR0 holds this enqueue

enq: RB - contention

Serializes OSM rollback recovery operations

enq: RF - synch: per-SGA Broker metadata

Ensures r/w atomicity of DG configuration metadata per unique SGA

enq: RF - synchronization: critical ai

Synchronizes critical apply instance among primary instances

enq: RF - new AI

Synchronizes selection of the new apply instance

enq: RF - synchronization: chief

Anoints 1 instance's DMON (Data Guard Broker Monitor) as chief to other instance's DMONs

enq: RF - synchronization: HC master

Anoints 1 instance's DMON as health check master

enq: RF - synchronization: aifo master

Synchronizes critical apply instance failure detection and failover operation

enq: RF - atomicity

Ensures atomicity of log transport setup

Enqueue Type

Description

enq: RN - contention

Coordinates nab computations of online logs during recovery

enq: RO - contention

Coordinates flushing of multiple objects

enq: RO - fast object reuse

Coordinates fast object reuse

enq: RP - contention

Enqueue held when resilvering is needed or when data block is repaired from mirror

enq: RS - file delete

Lock held to prevent file from accessing during space reclamation

enq: RS - persist alert level

Lock held to make alert level persistent

enq: RS - write alert level

Lock held to write alert level

enq: RS - read alert level

Lock held to read alert level

enq: RS - prevent aging list update

Lock held to prevent aging list update

enq: RS - record reuse

Lock held to prevent file from accessing while reusing circular record

enq: RS - prevent file delete

Lock held to prevent deleting file to reclaim space

enq: RT - contention

Thread locks held by LGWR, DBW0, and RVWR (Recovery Writer, used in Flashback Database operations) to indicate mounted or open status

enq: SB - contention

Synchronizes logical standby metadata operations

enq: SF - contention

Lock held for recovery when setting sender for AQ e-mail notifications

enq: SH - contention

Enqueue always acquired in no-wait mode; should seldom see this contention

enq: SI - contention

Prevents multiple streams table instantiations

enq: SK - contention

Serialize shrink of a segment

enq: SQ - contention

Lock to ensure that only one process can replenish the sequence cache

enq: SR - contention

Coordinates replication / streams operations

enq: SS - contention

Ensures that sort segments created during parallel DML operations aren't prematurely cleaned up

Enqueue Type

Description

enq: ST - contention

Synchronizes space management activities in dictionary-managed tablespaces

enq: SU - contention

Serializes access to SaveUndo Segment

enq: SW - contention

Coordinates the 'alter system suspend' operation

enq: TA - contention

Serializes operations on undo segments and undo tablespaces

enq: TB - SQL Tuning Base Cache Update

Synchronizes writes to the SQL Tuning Base Existence Cache

enq: TB - SQL Tuning Base Cache Load

Synchronizes writes to the SQL Tuning Base Existence Cache

enq: TC - contention

Lock held to guarantee uniqueness of a tablespace checkpoint

enq: TC - contention2

Lock during setup of a unique tablespace checkpoint in null mode

enq: TD - KTF dump entries

KTF dumping time/scn mappings in SMON_SCN_TIME table

enq: TE - KTF broadcast

KTF broadcasting

enq: TF - contention

Serializes dropping of a temporary file

enq: TL - contention

Serializes threshold log table read and update

enq: TM - contention

Synchronizes accesses to an object

enq: TO - contention

Synchronizes DDL and DML operations on a temp object

enq: TQ - TM contention

TM access to the queue table

enq: TQ - DDL contention

DDL access to the queue table

enq: TQ - INI contention

TM access to the queue table

enq: TS - contention

Serializes accesses to temp segments

enq: TT - contention

Serializes DDL operations on tablespaces

enq: TW - contention

Lock held by one instance to wait for transactions on all instances to finish

Enqueue Type

Description

enq: TX - contention

Lock held by a transaction to allow other transactions to wait for it

enq: TX - row lock contention

Lock held on a particular row by a transaction to prevent other transactions from modifying it

enq: TX - allocate ITL entry

Allocating an ITL entry in order to begin a transaction

enq: TX - index contention

Lock held on an index during a split to prevent other operations on it

enq: UL - contention

Lock held used by user applications

enq: US - contention

Lock held to perform DDL on the undo segment

enq: WA - contention

Lock used for recovery when setting watermark for memory usage in AQ notifications

enq: WF - contention

Enqueue used to serialize the flushing of snapshots

enq: WL - contention

Coordinates access to redo log files and archive logs

enq: WP - contention

Enqueue to handle concurrency between purging and baselines

enq: XH - contention

Lock used for recovery when setting No Proxy Domains for AQ HTTP notifications

enq: XR - quiesce database

Lock held during database quiesce

enq: XR - database force logging

Lock held during database force logging mode

enq: XY - contention

Lock used by Oracle Corporation for internal testing


Pages

Powered by Movable Type 6.3.2

About this Archive

This page is an archive of entries from March 2011 listed from newest to oldest.

February 2011 is the previous archive.

April 2011 is the next archive.

回到 首页 查看最近文章或者查看所有归档文章.