본문 바로가기

분류 전체보기

Attention is All You Need 1. 논문 제목 : Attention is All You Need(Transformer) 2. 발표한 학회 : NIPS 2017 3. 논문의 핵심 키워드 : attention mechanisms, multi-head attention, self-attention, Encoder-Decoder 4. 논문요약 : 기존 RNN 기반의 언어 모델들이 가지고 있는 parallelization, vanishing gradient등의 문제점들을 해결하기 위해서 attention mechanism만을 활용한 Encoder-Decoder 모델을 처음 선보인 논문. NLP분야 전반적으로 각 task들을 해결하기 위해 주로 RNN, CNN 기반의 모델들이 활용되는 패러다임을 attention 기반의 모델로 크게 바꾸는 신호.. 더보기
Universal Language Model Fine-tuning for Text Classification 1. 논문 제목 : Universal Language Model Fine-tuning for Text Classification (ULMFiT) 2. 발표한 학회 : ACL 2018 3. 논문의 핵심 키워드 : inductive transfer learning, pretraining, fine-tuning, language model 4. 논문요약 : BERT, GPT부터 RoBERTa, T5등 최신 Language Model들이 모두 차용하는 pretraining-finetuning의 schema를 정리하여 기틀을 닦은 논문 5. 논문에서 소개하는 이전 SOTA / 기존 방법들의 한계: 1) 컴퓨터 비젼 분야에서는 ImageNet, MS-COCO등의 범용적인 데이터셋에 모델을 pretrain하고, 필요.. 더보기
AJAX & Promise & Fetch API - AJAX stands for Asynchronous JavaScript and XML. - AJAX is not a technology in itself, but the use of XMLHttpRequest to communicate with servers. - Using AJAX, we can update web pages asynchronously, meaning updating parts of a page without reloading the whole page. - This concept allows apps to be faster and more responsive to user actions. - By creating a XMLHttpRequest object which manages .. 더보기
HTML Document Object Model(DOM) & JavaScript Events - The Document Object Model (DOM Model) is the standard object model and API for HTML. - An object model for HTML defines elements as objects in an object hierarchy, properties for all HTML elements, methods for all HTML elements, and events for all HTML elemnts. - An API for JS can add/change/remove HTML elements/attributes/events and CSS styles. - When a web page is loaded, a browser creates a.. 더보기
Assembly Language Fundamentals - Main PROC / INVOKE ExitProcess,0 / Main ENDP 는 기본적으로 적어야하는 코드 틀이다. - INVOKE ExitProcess,0을 시행하게 되면 OS 내에 특정 부분으로 JUMP하게 된다. - 변수를 선언하기 위해서 .data와 .code로 구분하게 된다. - Assembly 언어에서는 numeric literal을 정의하기 위해서 숫자 뒤에 d(decimal), h(hexadecimal), b(binary), q(octal)을 붙여줘야한다. - 이때, default는 decimal이다. - numeric literal 사이의 연산자에는 +, -, *, /, MOD 등이 있다. - When we use variables, we should not use reserved.. 더보기
JavaScript Syntax(2) - Modules in JavaScript are called Functions, and pre-defined functions that belong to objects are called Methods. - Like def in Python, we should explicitly write function when defining a function in JS. - Like C++, it doesn't matter where you define your function. (It's okay to define it on the bottom) - Like C++, there exist two ways to pass parameters, 'Pass-by-value', and 'Pass-by-reference.. 더보기
JavaScript Syntax(1) - JavaScript unlike HTML, CSS is a programming language that enhances functionality and appearances of Web pages. - JavaScript is the standard of client-side scripting languages and makes web pages more dynamic and interactive. - Like CSS, programmers could write JS code inside the HTML file withing the tag, or create a separate js file and link it. - We should use the second method, because pro.. 더보기
x86 Processor Architecture - IA-32 is a basic model we could use to understand about computer architectures before learning about more advanced 64 bit processors. - As mentioned in previous articles, the job of the CPU is to succesfully perform the 'fetch -> decode -> execute cycle'. - Every operation is synchronized by the clock. - Each step in instruction cycle takes one clock cycle. - Specifically, each process is proc.. 더보기

티스토리툴바