Data-ServicesComponent
ActivePublic

Milestones

Members

  • This project does not have any members.
  • View All

Details

Description

Data services available for use on Cloud-Services.

  • Wiki Replicas - redacted copies of the Wikimedia wiki's production databases
  • Wikimedia Dumps - public hosting of full text dumps from Wikimedia wikis and other datasets in Cloud (see also: Dumps-Generation for generation of dumps themselves)
  • Shared Storage - NFS storage for cross-VM and cross-project use

Recent Activity

Mon, Sep 29

FCeratto-WMF closed T404566: Prepare and check storage layer for tokwiki, a subtask of T404570: [wikireplicas] Create views for new wiki tokwiki, as Resolved.
Mon, Sep 29, 6:33 PM · Data-Services, cloud-services-team
FCeratto-WMF closed T404699: Prepare and check storage layer for mswikiquote, a subtask of T404703: [wikireplicas] Create views for new wiki mswikiquote, as Resolved.
Mon, Sep 29, 6:32 PM · Data-Services, cloud-services-team

Fri, Sep 26

Gehel removed a project from T404473: Unexpected error "Subquery returns more than 1 row" on wiki replicas: Data-Platform-SRE.
Fri, Sep 26, 2:02 PM · Community-Tech, Multiblocks, Data-Services, cloud-services-team
Gehel edited projects for T387419: Create wiki replicas views for globaljsonlinks tables, added: Data-Platform-SRE (2025.09.26 - 2025.10.17); removed Data-Platform-SRE (2025.09.05 - 2025.09.26).
Fri, Sep 26, 1:50 PM · Data-Platform-SRE (2025.09.26 - 2025.10.17), Essential-Work, Data-Engineering-Radar, Data-Services, cloud-services-team, Data-Persistence, Data-Engineering
Gehel edited projects for T382607: Decision request - Who runs wikireplicas cookbooks, added: Data-Platform-SRE (2025.09.26 - 2025.10.17); removed Data-Platform-SRE (2025.09.05 - 2025.09.26).
Fri, Sep 26, 1:50 PM · Data-Platform-SRE (2025.09.26 - 2025.10.17), cloud-services-team (FY2025/26-Q1), Data-Services, Data-Persistence, Cloud Services Proposals
Maintenance_bot removed a project from T405728: JobUnavailable Reduced availability for job maintain_dbusers_eqiad in cloud@eqiad: Patch-For-Review.
Fri, Sep 26, 1:32 PM · Data-Services, cloud-services-team
taavi closed T405728: JobUnavailable Reduced availability for job maintain_dbusers_eqiad in cloud@eqiad as Resolved.
Fri, Sep 26, 1:00 PM · Data-Services, cloud-services-team
gerritbot added a comment to T405728: JobUnavailable Reduced availability for job maintain_dbusers_eqiad in cloud@eqiad.

Change #1191675 merged by Majavah:

[operations/puppet@production] P:wmcs: maintain-dbusers: Remove accounts names from Prometheus labels

https://gerrit.wikimedia.org/r/1191675

Fri, Sep 26, 12:55 PM · Data-Services, cloud-services-team
gerritbot added a project to T405728: JobUnavailable Reduced availability for job maintain_dbusers_eqiad in cloud@eqiad: Patch-For-Review.
Fri, Sep 26, 11:55 AM · Data-Services, cloud-services-team
gerritbot added a comment to T405728: JobUnavailable Reduced availability for job maintain_dbusers_eqiad in cloud@eqiad.

Change #1191675 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] P:wmcs: maintain-dbusers: Remove accounts names from Prometheus labels

https://gerrit.wikimedia.org/r/1191675

Fri, Sep 26, 11:55 AM · Data-Services, cloud-services-team
taavi claimed T405728: JobUnavailable Reduced availability for job maintain_dbusers_eqiad in cloud@eqiad.
Fri, Sep 26, 11:53 AM · Data-Services, cloud-services-team

Thu, Sep 25

SD0001 closed T404473: Unexpected error "Subquery returns more than 1 row" on wiki replicas as Resolved.

Working fine now. Thanks!

Thu, Sep 25, 3:20 PM · Community-Tech, Multiblocks, Data-Services, cloud-services-team
fnegri added a comment to T404473: Unexpected error "Subquery returns more than 1 row" on wiki replicas.

I found the issue, I did run maintain-views too soon, before Puppet did synchronize the new yaml definition to all servers. I have re-run it on all clouddbs just to be sure all are in sync.

Thu, Sep 25, 8:43 AM · Community-Tech, Multiblocks, Data-Services, cloud-services-team

Wed, Sep 24

Ottomata edited projects for T404473: Unexpected error "Subquery returns more than 1 row" on wiki replicas, added: Data-Platform-SRE; removed Data-Engineering.
Wed, Sep 24, 4:02 PM · Community-Tech, Multiblocks, Data-Services, cloud-services-team
fnegri added a comment to T404473: Unexpected error "Subquery returns more than 1 row" on wiki replicas.

@SD0001 apologies, I think I did something wrong yesterday and the change was not applied to clouddb1017. I re-applied it now, and your query is now working fine.

Wed, Sep 24, 2:14 PM · Community-Tech, Multiblocks, Data-Services, cloud-services-team
SD0001 added a comment to T404473: Unexpected error "Subquery returns more than 1 row" on wiki replicas.

All done for an-redacteddb1001. Thanks, all.

Wed, Sep 24, 11:27 AM · Community-Tech, Multiblocks, Data-Services, cloud-services-team
BTullis reassigned T404473: Unexpected error "Subquery returns more than 1 row" on wiki replicas from BTullis to SD0001.
Wed, Sep 24, 8:41 AM · Community-Tech, Multiblocks, Data-Services, cloud-services-team
BTullis added a comment to T404473: Unexpected error "Subquery returns more than 1 row" on wiki replicas.

All done for an-redacteddb1001. Thanks, all.

Wed, Sep 24, 8:41 AM · Community-Tech, Multiblocks, Data-Services, cloud-services-team

Tue, Sep 23

taavi closed T238497: Denormalize user_groups to contain actor information as Declined.

I'm declining since maintaining additional "real" columns would be very complicated for the relatively little benefit they give.

Tue, Sep 23, 6:05 PM · cloud-services-team, Data-Services
taavi closed T215445: comment and actor view challenges for Cloud Services as Resolved.

As far as I can tell there is no plan to change the current subview setup. So closing to avoid this lingering on various workboards indefinitely.

Tue, Sep 23, 6:04 PM · cloud-services-team, Data-Services
taavi moved T404473: Unexpected error "Subquery returns more than 1 row" on wiki replicas from Inbox to Watching on the cloud-services-team board.
Tue, Sep 23, 5:27 PM · Community-Tech, Multiblocks, Data-Services, cloud-services-team
Maintenance_bot removed a project from T404473: Unexpected error "Subquery returns more than 1 row" on wiki replicas: Patch-For-Review.
Tue, Sep 23, 4:32 PM · Community-Tech, Multiblocks, Data-Services, cloud-services-team
fnegri reassigned T404473: Unexpected error "Subquery returns more than 1 row" on wiki replicas from fnegri to BTullis.

Change applied to all clouddbs, @BTullis handing over to you for an-redacteddb.

Tue, Sep 23, 4:23 PM · Community-Tech, Multiblocks, Data-Services, cloud-services-team
fnegri changed the status of T404473: Unexpected error "Subquery returns more than 1 row" on wiki replicas from Open to In Progress.

I'm applying the change in https://gerrit.wikimedia.org/r/1187896 to all clouddb* hosts with:

Tue, Sep 23, 4:18 PM · Community-Tech, Multiblocks, Data-Services, cloud-services-team
gerritbot added a comment to T404473: Unexpected error "Subquery returns more than 1 row" on wiki replicas.

Change #1187896 merged by FNegri:

[operations/puppet@production] maintain-views: fix filtering for actor view

https://gerrit.wikimedia.org/r/1187896

Tue, Sep 23, 4:07 PM · Community-Tech, Multiblocks, Data-Services, cloud-services-team

Mon, Sep 22

FCeratto-WMF changed the status of T404566: Prepare and check storage layer for tokwiki, a subtask of T404570: [wikireplicas] Create views for new wiki tokwiki, from Open to In Progress.
Mon, Sep 22, 9:22 AM · Data-Services, cloud-services-team
fnegri moved T404570: [wikireplicas] Create views for new wiki tokwiki from Backlog to Wiki replicas on the Data-Services board.
Mon, Sep 22, 9:05 AM · Data-Services, cloud-services-team
fnegri moved T404703: [wikireplicas] Create views for new wiki mswikiquote from Backlog to Wiki replicas on the Data-Services board.
Mon, Sep 22, 9:05 AM · Data-Services, cloud-services-team
fnegri triaged T404703: [wikireplicas] Create views for new wiki mswikiquote as Medium priority.
Mon, Sep 22, 9:05 AM · Data-Services, cloud-services-team
fnegri triaged T404570: [wikireplicas] Create views for new wiki tokwiki as Medium priority.
Mon, Sep 22, 9:05 AM · Data-Services, cloud-services-team
fnegri added a subtask for T404570: [wikireplicas] Create views for new wiki tokwiki: T404566: Prepare and check storage layer for tokwiki.
Mon, Sep 22, 9:05 AM · Data-Services, cloud-services-team
fnegri added a subtask for T404703: [wikireplicas] Create views for new wiki mswikiquote: T404699: Prepare and check storage layer for mswikiquote.
Mon, Sep 22, 9:04 AM · Data-Services, cloud-services-team
fnegri added a project to T404473: Unexpected error "Subquery returns more than 1 row" on wiki replicas: Data-Engineering.
Mon, Sep 22, 8:53 AM · Community-Tech, Multiblocks, Data-Services, cloud-services-team

Tue, Sep 16

akosiaris changed the status of T401295: Decide how to use the new clouddb hosts (clouddb102[2-5]) from Open to Stalled.

Setting to stalled, while we figure out the exact details of this one.

Tue, Sep 16, 2:01 PM · Data-Platform-SRE, Data-Services, Data-Persistence, cloud-services-team
Maintenance_bot created T404703: [wikireplicas] Create views for new wiki mswikiquote.
Tue, Sep 16, 11:10 AM · Data-Services, cloud-services-team

Mon, Sep 15

Maintenance_bot created T404570: [wikireplicas] Create views for new wiki tokwiki.
Mon, Sep 15, 9:22 AM · Data-Services, cloud-services-team

Fri, Sep 12

gerritbot added a project to T404473: Unexpected error "Subquery returns more than 1 row" on wiki replicas: Patch-For-Review.
Fri, Sep 12, 8:25 PM · Community-Tech, Multiblocks, Data-Services, cloud-services-team
gerritbot added a comment to T404473: Unexpected error "Subquery returns more than 1 row" on wiki replicas.

Change #1187896 had a related patch set uploaded (by SD0001; author: SD0001):

[operations/puppet@production] maintain-views: fix filtering for actor view

https://gerrit.wikimedia.org/r/1187896

Fri, Sep 12, 8:25 PM · Community-Tech, Multiblocks, Data-Services, cloud-services-team
SD0001 added a project to T404473: Unexpected error "Subquery returns more than 1 row" on wiki replicas: Multiblocks.

Simplified query with the same issue: SELECT actor_id from actor.

Fri, Sep 12, 8:11 PM · Community-Tech, Multiblocks, Data-Services, cloud-services-team
JJMC89 moved T404473: Unexpected error "Subquery returns more than 1 row" on wiki replicas from Backlog to Wiki replicas on the Data-Services board.
Fri, Sep 12, 7:53 PM · Community-Tech, Multiblocks, Data-Services, cloud-services-team
taavi edited projects for T404473: Unexpected error "Subquery returns more than 1 row" on wiki replicas, added: Data-Services; removed Toolforge.
Fri, Sep 12, 7:53 PM · Community-Tech, Multiblocks, Data-Services, cloud-services-team
Gehel moved T402283: Title of suppressed recentchanges entries can be viewed through the wiki replicas from Done to Reported on the Data-Platform-SRE (2025.09.05 - 2025.09.26) board.
Fri, Sep 12, 2:34 PM · Essential-Work, Data-Platform-SRE (2025.09.05 - 2025.09.26), Data-Engineering (Q1 FY25/26 July 1st - September 30th), SecTeam-Processed, Data-Persistence, cloud-services-team, Vuln-Infoleak, Data-Services, Privacy Engineering, Trust and Safety Product Team, Security

Thu, Sep 11

Gehel added a project to T387419: Create wiki replicas views for globaljsonlinks tables: Essential-Work.
Thu, Sep 11, 12:13 PM · Data-Platform-SRE (2025.09.26 - 2025.10.17), Essential-Work, Data-Engineering-Radar, Data-Services, cloud-services-team, Data-Persistence, Data-Engineering
Gehel added a project to T402283: Title of suppressed recentchanges entries can be viewed through the wiki replicas: Essential-Work.
Thu, Sep 11, 12:11 PM · Essential-Work, Data-Platform-SRE (2025.09.05 - 2025.09.26), Data-Engineering (Q1 FY25/26 July 1st - September 30th), SecTeam-Processed, Data-Persistence, cloud-services-team, Vuln-Infoleak, Data-Services, Privacy Engineering, Trust and Safety Product Team, Security

Wed, Sep 10

Ottomata added a project to T402145: Create view for betafeatures_user_counts table in wiki replicas: Data-Platform-SRE.
Wed, Sep 10, 4:13 PM · Data-Platform-SRE, Data-Engineering, BetaFeatures, cloud-services-team, Data-Services
Ottomata moved T59617: Make watchlist table available as curated foo_p.watchlist_count on labsdb from Incoming (new tickets) to Backlog on the Data-Engineering board.
Wed, Sep 10, 4:02 PM · Data-Engineering, cloud-services-team, Data-Services, Patch-For-Review, DBA

Tue, Sep 9

fnegri closed T404090: [wikireplicas] clouddb1015 replication lag when applying ALTER TABLE as Resolved.

Replication is back in sync.

Tue, Sep 9, 3:34 PM · cloud-services-team (FY2025/26-Q1), Data-Services
sbassett set Author Affiliation to product on T402283: Title of suppressed recentchanges entries can be viewed through the wiki replicas.
Tue, Sep 9, 2:47 PM · Essential-Work, Data-Platform-SRE (2025.09.05 - 2025.09.26), Data-Engineering (Q1 FY25/26 July 1st - September 30th), SecTeam-Processed, Data-Persistence, cloud-services-team, Vuln-Infoleak, Data-Services, Privacy Engineering, Trust and Safety Product Team, Security
fnegri moved T404090: [wikireplicas] clouddb1015 replication lag when applying ALTER TABLE from Backlog to In progress on the cloud-services-team (FY2025/26-Q1) board.
Tue, Sep 9, 2:28 PM · cloud-services-team (FY2025/26-Q1), Data-Services
fnegri triaged T404090: [wikireplicas] clouddb1015 replication lag when applying ALTER TABLE as High priority.
Tue, Sep 9, 2:28 PM · cloud-services-team (FY2025/26-Q1), Data-Services