More about the world of Ensembl

Who is using us?

During the long flight from London to Seoul, where I'm now giving workshops, I had time to do some analysis.

We have recently changed the way we analyse traffic on our website. Urchin Software allows us to pinpoint access to our site with more accuracy. (In a previous post, I showed data where some domains were excluded in the representation).

This month's data (June 2009) is more comprehensive and can be shown with the following heatmap. Again, dark-coloured countries show more use than lightly shaded ones. We can now detect some low level traffic in the African continent missed in previous analyses.

Following a recent thread, I normalised these data taking into account population. (Did I mention how long the flight from London to Seoul actually is?) This puts the Netherlands as the country with the highest ratio of Ensembl users normalised by population, followed by Sweden, Switzerland, the United Kingdom, Finland, Belgium, Iceland, Germany, Denmark and Singapore (the first non European country in the list) pushing the United States (excluding traffic from the US mirror) to the 15th position.

On another matter, I'm glad to see that our user base in this country is well established. Apart from the obvious Seoul, we can also see access from Taejon, Kwangju, Taegu, Pusan, Inchon, Pohang and up to 29 locations in the country, and following these workshops we hope this will increase further.

As they say over here 안녕

Xosé

Go, go, go .... the new Ensembl ontology database and API

Starting with release 55 of Ensembl we provide an ensembl_ontology database. It replaces the older ensembl_go database which used to be loaded straight from the public table dumps provided by the Gene Ontology group (and hence wasn't really an Ensembl database to start with). The associated API is now part of the Ensembl Core API, which should make working with GO terms in Ensembl more straightforward than it was in the past. Available methods include, amongst others, fetching all parent or child terms of a given GO term and fetching all genes, transcripts or translations annotated with a given GO term.

More detailed documentation on both database and API can be found at ensembl/misc-scripts/ontology/README.

Credit for developing the ensembl_ontology database and API goes to Andreas Kahari of the Ensembl Software team.

Colourful gene trees

We have some news for the forthcoming ensembl release. We have added a few more display options for our gene trees. It will be possible to colour the background of the trees based on the taxonomy. It will be much easier to locate orthologues or paralogues in a given clade. For people who prefer more subtle colouring, they can choose to colour the branches instead.

It will also be possible to automatically collapse all the genes for a given clade. In the example shown in this figure, glires and diptera are collapsed. Moreover, the new version will also allow you to hide fish genes for instance or even all genes from low-coverage genomes.

All these options are configurable from the configure panel available through the 'configure page' link in the left panel.

Ensembl Workshops in July

The summer is a time for research institutions to focus on research (or go on holiday). This causes a decrease in our Ensembl Outreach and Training (at least, workshop-wise!) However, we are giving a few workshops in July.

2 July: Internal Developer's Course (let's see what the developers themselves have to say about browsing Ensembl!)
6-7 July: Browser modules in EBI roadshow, Poznan Summerschool of Bioinformatics, Poland
9-10 July: Browser workshop in Seoul, Korea (Korea National Institute of Health)
13 July: Browser workshop in Daejeon, Korea (KRIBB)
13 July: Browser workshop at Wellcome Trust Genome Campus, for campus employees
14-15 July: Browser workshop in Pusan, Korea (Pusan National University)

Don't forget, we have loads of training videos on our tutorials page (and YouTube channel!)

US Mirror

Ensembl is pleased to announce the release of its West Coast US mirror (uswest.ensembl.org). This is a full mirror of the current Ensembl 54 release. We are providing this mirror to improve performance for users in the US, particularly on the West coast. It includes full search, BioMart and BLAST support (BLAST searching is actually run at Sanger with results passed back to the mirror).

This mirror is managed directly by the Ensembl web team, and we will aim to update it along with the main site, to keep it current. Credit for gettting this mirror up goes to James Smith and Eugene Bragin from the web team, with support from the Sanger systems team, particularly Peter Clapham, John Nicholson and Dave Holland.

Future plans: We will improve the mirror in the near future by allowing users to switch between the main and mirror site. Currently, we do not suggest logging in to the mirror. All user data must be retrieved by the main site at the Wellcome Trust Genome Campus. Speed is optimal if login is not used, however this will be improved in the future.

Many thanks,
The Ensembl Team

A view of the world from Ensembl

Following our recent post updating you with feedback from our recent user survey, I wanted to share with you the following data. I've been data mining our web logs to get a picture of worldwide access to Ensembl (this is page impressions) and I put them on a heat map representing access to our browser (this doesn't include either BioMart or direct access of our public databases). I filtered out commercial IP addresses (i.e. all those .com, and .net) to simplify the analysis. Countries are shaded according to how much they accessed the Ensembl browser in the month of May. Dark countries use the Ensembl browser more, light countries, less. And this is what you get:

Can you see your country here? If not, consider running a browser workshop in order to join our wide community of Ensembl users!

This map should help us to monitor the success of our workshops (at this moment I'm writing this from Venezuela which hopefully should appear in future heat maps!) Although there are still some gaps in the map, we are happy to see that Ensembl is used globally and our efforts are recompensed. Training leads to more in depth understanding of how to use the browser, and we think our worldwide workshops are helpful in using Ensembl to guide and enhance research.

Greetings from Venezuela, or as you would say here... ¡Hasta la próxima

Xosé

Thanks for your feedback!

Dear all,

Thanks to all of you who participated in our Ensembl browser survey, which closed at well over 150 respondents. We appreciate hearing from users worldwide, and our responses came from Europe, the Americas, Asia, India, New Zealand and Australia.

We will be able to move forward based on the feedback, in order to make the browser smoother and easier to use.

Some of the things we learned.

How often do you use Ensembl?

The majority of our users (83%) access the Ensembl browser multiple times per week. (See pie chart above). A similar survey run by Ensembl in 2007 reflected a much lower figure: only 32% of users browsed as frequently, with the majority of respondents accessing the browser once a month or less. Genome browsers appear to have moved from being tentative, browse-for-interest curiousities to comprehensive resources that can be integrated into daily research.

We were pleased to see our users are accessing many types of Ensembl data, from transcripts to variations to comparative genomics. Most respondents found the tabs in our new interface to be intuitive, so we won't change those! Many users scroll through views using the buttons on each page, so again, we will keep those in place.

More of you are finding the Pre! and Archive! sites helpful. And 65% of users voted that the most important aspect was good information.

Things we will work on?

In addition to custom data and user upload advances, we are focusing on graphical display of our comparative data. These improvements are already in our pipeline. We did get some new ideas from the survey, for example, some of our respondents could not find the Pre! or Archive! pages. We will think about how to make these links more obvious.

Speed of the browser could be faster. We will keep you posted on the Ensembl mirror on the US west coast.

Finally, more users are finding our help pages and videos, including a new YouTube channel. and we will be constantly adding to the tutorials page. If you have ideas about videos or tutorials you would like to see, please send your comments to our helpdesk. We find it immensely helpful to hear feedback from our user community.

Thanks again for the view of Ensembl from our users! We will be polling again next year.

The Ensembl Team

Give us your feedback!

Hello all,

We are curious as to how our users are finding the current Ensembl web browser. We opened a survey in the current Ensembl (version 54). The survey will close in one week's time (Friday, 5 June). Thank you to everyone who has already entered their feedback.

For users who have not yet replied to the survey, we ask that you spare 10 minutes or so of your time to do so. Please give us your thoughts and feedback by clicking on the link below:

http://tinyurl.com/cv67vs

The feedback centers on the web browser, specifically the new interface launched in Nov, 2008.

Many thanks for your time.

Regards,
The Ensembl Team

Ensembl Events in June 2009

For June we have the following Ensembl events:

4-5 June : Browser workshop at the University of Cambridge, Cambridge, UK
11 June: Demo for the National Genetics Reference Lab, Manchester, UK
15-16 June: Browser workshop at the Facultad de Ciencias de la Universidad de Los Andes, Mérida, Venezuela
18-19 June: Developers workshop at the Facultad de Ciencias de la Universidad de Los Andes, Mérida, Venezuela
22 June: Browser workshop for NHS Molecular Genetics Laboratories at Liverpool Women's Hospital, Liverpool, UK
24-26 June: Ensembl module in the Bioinformatics for Vascular Biology course at the EBI, Hinxton, UK
29 June: Browser workshop at the Leibniz Institute for Zoo and Wildlife Research, Berlin, Germany

For details about these and other upcoming events, please have a look at the complete list of Ensembl training events.

Pre, Archive, and Vega downtime

Next week, 18-24 May, an upgrade is scheduled that will affect the following sites:

Archive sites versions 48-53 (Dec 2007-May 2009) (Downtime will start Monday 18 May)

BLAST and User Upload on Pre! sites (Wednesday 20 May)

BLAST and User Upload on the Vega site

We ask our users to plan to use these sites after the upgrade, if possible.

Regards,
The Ensembl Team

Ensembl 55

We are currently working on our next release which is due at the end of June 2009 and will contain the following:

Data

Human GRCh 37
We will be releasing a new genebuild for human based on the lastest assembly GRCh37 from the Genome Reference Consortium. A preliminary version of this assembly is available now in Ensembl Pre! Due to the new assembly we will have:

• Updated repeat masking
• New probeset mappings
• cDNA update
• A new ensembl-vega merge delivering a new gene set
  

Wallaby
Ensembl 55 includes the 2X genome for Tammar Wallaby (Macropus eugenii), this will be a projection build similar to our other 2X species.

C. elegans
We will also include an import of the WormBase release WS200 database for C. elegans.

Anole lizard - A gene patch incorporating the gene set provided by Chris Ponting at Oxford University means that we have a new gene set for the green anole lizard (Anolis Carolinensis).

Mouse - The mouse cDNA alignments have been updated.

Zebrafinch - There will be an updated gene set for the 6X zebra finch genome.

Zebrafish - Non-coding RNAs will be added to the Zv8 zebrafish assembly and there will also be some changes to protein coding gene models and new repeats and expression patterns.

Core

Schema Changes

• Patch to update versions (patch_54_55_a.sql). * Add the missing types to go_xref (patch_54_55_b.sql).
• Add new table dependent_xref (will hold the dependencys for the xrefs, i.e. if an EMBL entry come from a uniprot entry this relationship will be in the table)( patch_54_55_d.sql).
• Add new tables for alternative splicing/transcript events (patch_54_55_c.sql).
• Add new column 'is_constitutive' to the exon table (patch_54_55_e.sql)
  

Xrefs
Xrefs will be run for Human, Macacca, Opossum, Chimp, Chicken, Dog and Mouse (including Fantom Xrefs).

Ontology database schema and tools
The ensembl_go_NN databases are no longer being built. Instead we are replacing this with the ensembl_ontology_NN database which may be connected to using the core API.

Assembly mapping
Some of the databases will contain mapping coordinates between current and previous assemblies:

• human: mapping from current GRCh37 to NCBI36, NCBI35 and NCBI34
• mouse: mapping from current NCBIM37 to NCBIM36, NCBIM35 and NCBIM34
  

Other changes

• API support for alternative transcripts/splicing events will be added
• API support for constitutive exons will be added
• Deprecated API modules will be removed
• All slices will be created using the new_fast method from the SliceAdaptor to improve performance
• seq_region seq edit support will be added. Seq_edits can already be stored and retrieved but these were not used in getting the sequence data. This will be changed so that "_rna_edit" attributes in the seq_region_attrib table will be used and the sequence changed.
• MySQL and FASTA dumps will be copied to Amazon Public Datasets project
  
• Gene name and xref projections
  
  

Mart

• New functional genomics mart * A new Probe section added to Ensembl mart
• New ontology mart
• Constitutive exon information will be re-added to Ensembl mart
  

Variation

• There will be a new human variation database generated by mapping NCBI36 coordinates to GRCh37 (using dbSNP 129)
• Illumina array data for SNP/CNV is to be added
• Transcript variations for Zebrafish and Zebrafinch will be reculated to include information from the new gene sets
• Schema change - added a call to get consequence_type
  

Functional genomics

• Human Regulatory Build will be updated using the GRCh37 assembly
• Probe alignment and transcript annotation for all species will migrate from the core datbases to the functional genomics databases, this includes Affymetrix, Illumina, Codelink and Phalanx
• Schema change, an is_current filed is to be added to the coord_system table
  

Comparative genomics

Alignments - The new human assembly means that the following alignments will be regenerated:

• 9 eutherian mammals EPO multiple alignments
• 31 eutherian mammals EPO multiple alignments
• 12 amniota vertbrates Pecan multiple alignments
• 4 catarrhini primate EPO multiple alignments
• Pairwise BLASTZ-NET alignments of human against each of the other 9 and 31 eutherian mammals
• Additional pairwise BLASTZ-NET alignments will be run for human-opossum, human-platypus, human- chicken and human-wallaby
• Translated BLAT-NET will be regenerated for human against fugu, X.tropicalis, C.intestinalis, C.savignyi, stickleback, medaka, chicken, zebrafish, tetraodon, zebrafinch and anole lizard
  

Synteny will be recalculated for: rat vs. huamn, chicken vs. human and human vs. macaque, dog, chimpanzee, platypus, opossum, mouse, orangutan, horse and cow

Homologies amd families

• 50 way GeneTrees and homologies with new/updated genebuilds and assemblies
• Clustering using hcluster_sg
• Multiple Sequence Alignments using consistency-based MCoffee meta-aligner (mafftgins + muscle + kalign + probcons) and new exon-skipping aware "skipper" algorithm.
• New 'putative gene split' and 'distant paralog' homology types
• Pairwise gene-based dN/dS calculations for high coverage species pairs
• Updated MCL families including all Ensembl transcript isoforms and newest Uniprot Metazoa
• Multiple sequence alignments with MAFFT
• Stable IDs for GeneTrees (ENSGT00550NNNNNNNNN) and MCL Families (ENSFM00550NNNNNNNNN).
  

SNPs and ancestral alleles

The new Ensembl release includes a new view for SNPs and other genomic variations. It shows the alignment of the polymorphic position together with 10 base pairs of sequence up- and downstream. The user can choose among all available multiple alignments. Polymorphic positions in the other species are also shown.

This is very useful for looking at ancestral alleles, especially in combination with our EPO alignments as they include the inferred ancestral sequence. Although dbSNP provide predicted ancestral alleles for human SNPs, these are based on the chimp sequence only. In several cases, the ancestral sequence inferred from the multiple alignment is in disagreement with the chimp sequence like in this example. Using multiple alignments gives better results and more confidence to the calls.

Release 54 and pre.ensembl.org

The Ensembl project is pleased to announce release 54 of Ensembl. Highlights of this release are:

• New Zv8 zebrafish assembly;
  
• Comparative alignment text displays for variations and regions;
  
• Ability to add personal notes to any Gene or Transcript.

For more information visit:

http://www.ensembl.org/info/website/news/index.html

Along side this release we are also releasing a new version of the pre site. This now includes:

• The GRCh37 human assembly released in February 2009, with preliminary analyses included;
• The callJacc3 marmoset assembly.
  

The eFG Array Mapping Environment

The Ensembl Functional Genomics (eFG) environment has been expanded to incorporate array mapping functionality. Historically, arrays from different vendors have been processed in similar, but non-identical ways due to differing array designs, with the output being stored in the core database. The 'arrays' environment unifies this process within the eFG database to provide a new standardised array mapping procedure for all array formats. This involves a two step process whereby probe sequences are aligned both to genomic and transcript sequences, and then subsequently transcripts are annotated with xrefs(DBEntries) dependant on the quality of the probe alignments around a given transcript locus.

The 'arrays' environment provides easily accessible and interactive command line functions to help run and administer the array mapping pipeline. Recent developments include broader array format support and multi-species capability, along with capture of much more detailed mapping information. This data has yet to be seen in the Ensembl browser, but from release 55 we will start redirecting the web displays to use the eFG data, with a view to developing a more detailed 'Probe' panel at some point later in the year.

We will endeavour to provide alignments and mappings of all popular arrays, for all others we invite you to try out the eFG 'arrays' environment. For more information check out(literally):

ensembl-functgenomics/docs/array_mapping.txt

Or see it online here.

If you have any questions, please mail ensembl-dev@ebi.ac.uk

Browser Training in Vienna on 22 May

The Ensembl Genome Browser project is pleased to announce a workshop on 22 May as a satellite meeting of the European Human Genetics Conference in Vienna, Austria. This full-day workshop is aimed at geneticists and life scientists, and will explore genes, variations, and comparative information using the browser's new interface released Nov, 2008. An introduction to large-scale data retrieval with BioMart will be included. We will also feature brief introductions into the European Genotype Archive (EGA) and the 1000 Genomes Project. The format of our browser workshops are described on our outreach page.

The course on 22 May is held at a central location- the Vienna University Computer Service.

The workshop is free, however limited places are available. Please register if you will be attending.

Ensembl Events in May 2009

For May we have the following Ensembl events:

29 April - 1 May: Ensembl Developers workshop at the University of Cambridge, Cambridge, UK
8 May: Browser workshop at Imperial College, London, UK
11-13 May: Ensembl module in the Wellcome Trust Open Door Workshop - Working with the Human Genome Sequence, Hinxton, UK
11-15 May: Ensembl module in the EBI hands-on training A walktrough EBI Bioinformatics Resources, Hinxton, UK
12-13 May: Ensembl module in the EBI roadshow at the Université Victor Segalen Bordeaux 2, Bordeaux, France
19-21 May: Ensembl module in the EBI roadshow at the Universidade de Santiago de Compostela, Santiago de Compostela, Spain
22 May: Browser workshop at the European Human Genetics Conference, Vienna, Austria
26 May: Ensembl Developers workshop at the VIB Flanders Interuniversity Institute of Biotechnology, Ghent, Belgium
27-28 May: Browser workshop at the Erasmus MC Molecular Medicine Postgraduate School, Rotterdam, The Netherlands

For details about these and other upcoming events, please have a look at the complete list of Ensembl training events.

Ensembl Genomes live!

Today the long-awaited Ensembl Genomes went live! This is a 'sister project' focusing on those species that aren't part of Ensembl, i.e. non-vertebrates. Please have a look at what the Ensembl Genomes team have to say about it themselves:

"We are delighted to announce the forthcoming release of Ensembl Bacteria, Ensembl Protists and Ensembl Metazoa, the first sites to be launched as part of the EBI's "Ensembl Genomes" project to extend the use of the Ensembl browser to non-vertebrate genomes.

These following site are available:

http://bacteria.ensembl.org
http://protists.ensembl.org
http://metazoa.ensembl.org

Additional sites for fungi and plants are in development and will be launched during the summer of this year.

In the Ensembl Genomes project, we are aiming to do two things: firstly to work with particular communities to support the bioinformatic analysis of genome-scale data; and secondly, to provide an integrative portal to data from species of scientific interest from across the taxonomic space. In pursuit of both these aims, we will re-use and extend the proven Ensembl software system, that has been developed by EBI and the Wellcome Trust Sanger Institute in the context of vertebrate genomics.

As with Ensembl, Ensembl Genomes will provide access to DNA and protein sequence, positional and functional annotation of protein-coding and non-protein coding genes, repeat analysis and other features and statistics. An interesting feature made available with the release of Ensembl Genomes is the inclusion of a multi-way comparative genomic analysis performed using a selection of species from bacteria to humans, and the production of gene trees showing the inferred ancestral relationships within deeply conserved protein families. Comparative resources are also provided at a narrower level (for example, DNA and protein-based analyses of individual bacterial clades). In partnership with collaborators, we are working on capturing gene expression, and population-scale variation data, in a number of contexts. More generally, we anticipate the ongoing enrichment of these resources through the integration of increasing quantities of high throughput data now becoming routinely available for all species.

Ensembl Genomes will provide access to data through the usual routes supported for vertebrate data; web-based browser, FTP site, programmatic API, DAS, and BioMart-style data warehouse; as well as text and sequence-based search.

We look forward to working with you as future producers and consumers of data. More information about the project is available at http://www.ensemblgenomes.org. We will be happy to receive any feedback you might wish to offer us at helpdesk@ensemblgenomes.org."

Ensembl Tips: Navigating the Browser

Though the overall response has been good, a few Ensembl users are finding it difficult to switch from the old interface to the new browser launched Nov, 2008. For those users, functionality has not been lost. You should still be able to do the same tasks as before in a faster interface.

We will post a series of tips to show you how to make the switch from the old interface to the new. If you still have trouble, please watch our video tutorial: Browsing Ensembl.

TIP: I want to use ExonView. Where is this now?

To view the full genomic sequences, exons and introns, go to any transcript. (Exons are transcript information. To see the exons page, go to a transcript tab, not the gene tab.) Click on the 'Exons' link under 'Sequence' at the left of any transcript page.

To show the full introns, click on 'Configure this page' at the left. Select 'Show full intronic sequence'. Click 'Save and Close' at the top right corner of the menu window.

Still can't find what you're looking for? Email us at helpdesk@ensembl.org.

Ensembl Events in April 2009

For April we have the following Ensembl events:

31 March - 1 April : Browser workshop at the IGC, Oeiras, Portugal
2-3 April: Ensembl Developers workshop at the IGC, Oeiras, Portugal
16 April: Demo for the National Genetics Reference Lab, Manchester, UK
17 April: Browser workshop at Imperial College, London, UK
22 April: Browser workshop at Imperial College, London, UK *postponed to 8 May*
27-29 April: Ensembl module in the EBI roadshow at the University of Iceland, Reykjavik, Iceland
27-29 April: BioMart module in the EBI hands-on training Programmatic Access to Biological Databases (Java), Hinxton, UK
29 April - 1 May: Ensembl Developers workshop at the University of Cambridge, Cambridge, UK

For details about these and other upcoming events, please have a look at the complete list of Ensembl training events.

New Release (53)!

Ensembl just updated the live site and underlying databases to
version 53.

Some new features include 'Active Tracks' and a searchable 'Configure this page'!


Go to any region of the chromosome.

Click 'Configure this page' at the left.

'Active tracks' allows you to see (and deselect) all tracks that are turned on.

'Search display' allows you to search for tracks in the menus. In this example, we searched for UniProt. Tracks from different menus appear.

For more updates, including new species, variations, and Amazon Web Services, see the news.