Scribd Logo
Upload

Welcome to Scribd!

  • Realtime Personalization and Recommendation With Stream Mining

    Uploaded by

    Mikio Braun
    5K views24 pages
    Talk given at the Berlin Buzzword 2014 conference, May 27, 2014, in Berlin, Germany In this talk I'll talk about using approximative data structures as an alternative to brute force scaling to build realtime personalization and profiling engines.

    Original Title

    Realtime Personalization and Recommendation with Stream Mining

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    Talk given at the Berlin Buzzword 2014 conference, May 27, 2014, in Berlin, Germany In this talk I'll talk about using approximative data structures as an alternative to brute force scaling to build realtime personalization and profiling engines.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    5K views24 pages

    Realtime Personalization and Recommendation With Stream Mining

    Uploaded by

    Mikio Braun
    Talk given at the Berlin Buzzword 2014 conference, May 27, 2014, in Berlin, Germany In this talk I'll talk about using approximative data structures as an alternative to brute force scaling to build realtime personalization and profiling engines.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    of 24

    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill

    Realtime personalization
    and recommendation with
    stream mining
    Mikio Braun
    @mikiobraun
    streamdrill
    Berlin Buzzwords 2014
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    Reacting to user behaior
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    !ow"
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    Making #rogress
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    $etting rid o% e&actness
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    'm#ortant users
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    Stream Mining to the rescue

    Stream mining algorithms(

    answer )stream *ueries+ with ,nite resources

    -.#ical e&am#les(

    how often does an item a##ear in a stream"

    how many distinct elements are in the stream"

    what are the top-k most frequent items"


    Continuous Stream of Data
    Bounded Resource
    Analyzer
    Stream Queries
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    !ea. !itters /a0k0a0 -o#1k2

    3ount actiities oer large item sets /millions4


    een more4 e0g0 'P addresses4 -witter users2

    'nterested in most actie elements onl.0


    Metwally, Agrawal, Abbadi, Efficient computation of Frequent and Top-k Elements in Data Streams, Internation Conference
    on Database Theory, 2!
    frank
    paul
    an
    feli!
    leo
    ale!
    "#
    "$
    %
    #
    &
    $
    'i!ed tables of counts
    Case "( element already in data base
    Case $( new element
    paul ")$ "$ "&
    nico ale! $
    nico &
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    3ount1Min Sketches

    Summarize histograms oer large %eature


    sets

    5ike bloom ,lters4 but better

    6uer.( -ake minimum oer all hash %unctions


    * * & *
    " " * $
    * $ * *
    * & # $
    * # & $
    $ ) # *
    " & + &
    * $ * %
    m bins
    n different
    ,as, functions
    -pdates for new entry
    Query result( "
    ./ Cormode and S/ Mut,ukris,nan/ "n impro#ed data stream summary$ The count-min sketch and its applications%
    0A123 $**), 4/ Algorit,m ##5"6( #%7+# 5$**#6 /
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    !ea. !itters oer -ime1
    7indow
    1ime
    DB

    8ee# old count


    data #eriodicall.

    9lternatie(
    :&#onential deca.
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    'nde&ing 3olumns
    page referer IP score
    8inde!/,tml google "&/$)/&$/"$ "*
    8post8"$& facebook )&/"&/)&/9+ :
    8inde!/,tml twitter 9/9$/$&/) 9
    8about/,tml google "&/$)/&$/"$ &
    /// /// /// //
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    Storing ;ata in -rends
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    Storing ;ata in -rends
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    Streamdrill

    Realtime 9nal.sis Solutions

    3ore :ngine(

    !ea. !itters counting < e&#onential deca.

    Instant counts = to#1k results oer time windows

    'n1Memor.

    written in Scala

    Modules

    Pro,ling and -rending

    Recommendations

    3ount ;istinct
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    Realtime >ser Pro,les
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    Realtime >ser Pro,les
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    Realtime >ser Pro,les
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    Realtime >ser Pro,les
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    Realtime user #ro,les

    Process 10k eents ? second on one


    machine

    -rack about 1 Million counts #er 1 $B

    Shard b. user %or higher accurac.


    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    Realtime Recommendation
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    Realtime Recommendation
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    Realtime Recommendation

    Pi#e in eents4 get recommendations

    Seed b. sam#ling #ast clicks

    9utomaticall. ada#ts oer time

    @umber o% alternaties /user based4 item


    based4 trends4 categories2
    Mikio Braun Realtime Personalization and Recommendation with Stream Mining streamdrill
    Summar.

    ;itch e&actness A 9##ro&imate with


    stream mining

    -rends to store all kinds o% counting


    structures

    React to realtime user behaior with


    managable resources

    You might also like