diff options
Diffstat (limited to 'notes')
| -rw-r--r-- | notes/twitter.org | 39 |
1 files changed, 37 insertions, 2 deletions
diff --git a/notes/twitter.org b/notes/twitter.org index 6aee604..c60238f 100644 --- a/notes/twitter.org +++ b/notes/twitter.org @@ -6,7 +6,7 @@ *** TODO Get up to speed on alerts / viz *** TODO Work with Philip to take the work / knowledge on load test cluster *** TODO Should I setup a 1:1 with him ? -*** TODO I don't want to split ownership +*** DONE I don't want to split ownership ** Mahak *** DONE Complete feedback *** TODO Should we do a 'one day cleanup' where we go through alerts / warnings and prune / fix ? @@ -18,8 +18,17 @@ For months I've been asking what's the consequence of having many partitions. Ei + IM with zookeeper + moving dataproducts We need to stop working on long term solution that might give us benefits, but we're not sure. We need to spend time to understand our current system. Without reporting we are blind. We have no idea how much resources are used, necessary, wasted, etc. +*** TODO I want internal PRR before we push queue model / placement +*** TODO Do you have a monthly sync. meeting with ads / ads prediction / search / MH ? +*** TODO I need to go faster for DLog migration, can I remove 10 hosts from ads prediction in atla ? That would help a lot at this point. +** Ravi +*** TODO DC tour as off-site ? +** Philip +*** DONE Work with Cory for the load test cluster +Talk with him so we can get complete documentation, runbooks, tools to monitor/check status, etc. * Projects -** OS 7 Migration +** [[https://jira.twitter.biz/browse/PUBSUB-17420][OS 7 Migration]] ++ [[https://docs.google.com/document/d/1_9JAwCB1BPa-IcYerG9w5VrDpA-swtGZG2-AgXUA8s0/edit][Doc]] for Mahak + update for an aggressive [[https://docs.google.com/document/d/1bv_tGtB2mNgaA5ToQLRseY0lzE2vlALTiM5NraQtSkE/edit#][timeline]]. + shared services are in progress ([[https://docs.google.com/document/d/1TVIIgc1mfvghj-cFUX0iINh6SgF1OmRSAvK_b4UjJKU/edit][doc]]) + progress for our various services, run ~[[file+emacs:../bin/tw-os7-report][tw-os7-report]]~. @@ -30,4 +39,30 @@ We need to stop working on long term solution that might give us benefits, but w SCHEDULED: <2017-02-17 Fri> * Notes ** Manhattan +3 different clusters: RO / RW / ZAC + +RO: 2 copies, no quorum. Configure the number of buckets (10K). Cluster has a set of mirror set. 2 nodes per mirror set. you can keep adding mirror set. + +coord / replicas are 2 processes running on each node. Query goes to coord, there's a consistent hashing to find which replicas has the data. Query both nodes in the mirror set, and fastest one replies. + +in RW there's a quorum. At least 2 nodes have to reply on the request. + +users create application (dataset) + + +QL: that's how they achieve strong consistency + +otherwise it's eventual consistency ** Hybrid Mesos for messaging services +** SRE Sync +*** Tasks ++ [X] Get exception for Messaging services + this was denied (?) ++ [ ] Write a doc on what needs to be done to bring kafka / kestrel up to date ++ [ ] [[https://jira.twitter.biz/browse/CLDS-1384][Mesos zombie ticket]] ++ [ ] Tickets for deploying faster on COORD +*** Notes ++ There's a disconnect in communication regarding the OS migration. ++ Find who's using Kafka and which library they use ++ what's next for MOPUB ? + |