Minerando a evolução da Web

É Tanta informação disponível…

Dentre as várias coisas que podemos assumir sobre Web 2.0 é o fato de que a tecnologia ficou barata e isso rendeu inúmeras soluções criativas com mais pessoas usando a internet. Os portais tornaram-se plataformas digitais e deixaram de ser responsáveis pela criação do conteúdo. A grande massa de conteúdo passou a vir dos próprios usuários e os “portais” passaram gerenciar esse conteúdo criado por seus usuários. Sem dúvidas, um grande volume de dados surgiu e nos influencia individualmente, afeta as empresas, tomadas de decisão, etc.

Informação prioritária

E é com tanta informação disponível que a gente deve priorizar o que faz sentido para nós. Essas plataformas digitais contemporâneas foram obrigadas a se tornar mais inteligentes porque com tanta informação disponível, como despertar desejos, criar audiência sobre qualquer coisa e como saber o que as pessoas pensam? Como fazer isso de forma rápida e assertiva? Indo – mas não se restringindo apenas – buscar conhecimento no mar de informação de forma automática, fazendo:

  1. seleção do conteúdo apropriado, com apoio dos sistemas de recomendações. Estes sistemas basicamente filtram e personalizam o conteúdo para os usuários.
  2. conhecimento através da análise o conteúdo, com apoio de sistemas que fazem mineração da Web (Web Mining). Basicamente aplica técnicas de mineração de dados (data mining) nos conteúdos nos documentos da internet (páginas, forums, blogs, redes sociais, etc).

Não só informação relevante mas sentimental e sábia

Os mecanismos da Web Mining se baseiam em áreas muito bem conhecidas; Information Retrieval (IR) e information Extraction (IE). Também faz uso de diversos algorítimos de mineração de textos para analisar esses diferentes tipos de dados (páginas, forums, blogs, redes sociais, etc) que não estruturados.

É possível pensar a a Web Mining em três formas (categorias):

  1. Web Structure Mining – conhecimento descoberto através dos hyperlinks e suas relações
  2. Web Usage Mining – identificação de modelos e padrões sobre usuários extraídos de logs de acesso e histórico de atividades
  3. Web Content Mining – extração de conhecimento do próprio conteúdo das páginas

Diferente dos sistemas de recomendação (que explora oportunidades nos relacionamentos relevantes), as técnicas de Web Mining exploram situações sobre, por exemplo, a partir de um lançamento de um produto, como determinar o sentimento de um cliente ou como saber a opinião pública sobre um governo. Os feedbacks distribuídos em lugares distintos como páginas, blogs e até no facebook são analisados com a técnica de Opinion Mining (Mineração de opniões) subcategoria da Web Content Mining.

Estamos falando da Web 3.0? Sem dúvidas.

Referências

  • Applying Supervised Opinion Mining Techniques on Online User Reviews – Ion SMEUREANU e Cristian BUCUR -2012
  • Recommender Systems: An Introduction. Dietmar Jannach, Markus Zanker, Alexander Felfernig, Gerhard Friedrich – 2012
  • Building Social Web Applications: Establishing Community at the Heart of Your Site, Bell, Gavin – 2009
  • Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data (Data-Centric Systems and Applications), Bing Liu – 2008
  • THE ARCHITECTURE OF INTELLIGENT CITIES – Integrating human, collective, and artificial intelligence to enhance knowledge and innovation, Nicos Komninos – 2006
  • Web 3.0: The Third Generation Web is Coming – Lifeboat Foundation Scientific Advisory Board. Nova Spivack. Extraído em 11/2013.

Introdução a SOA

Resolvi deixar disponível um workshop introdutório sobre SOA a pedidos de alguns colegas.

OBJETIVO

Iniciar conceitos de nível básico-médio sobre SOA. Expor o conhecimento fundamental necessário para um anti-rojeto, mostrando exemplos de construção de serviços e uma possível arquitetura simples, enxuta e bem fundamentada. Também idealizar mudanças na construção de produtos e formalizar o conceito SOA, dismistificando-o.

PÚBLICO

  • Gestores
  • Desenvolvedores
  • Estudantes
  • E aqueles que acreditam ter vivenciado um projeto SOA porque criou algum web service ou um projeto com vários serviços feitos com WCF” ou usam o barramento como sinônimo estrutural de SOA. (confusão mental sobre metodologia e tecnologia que inferioriza a proposta de SOA numa empresa e deixa o mercado ruim)

A apresentação é baseada na literatura SOA de Thomas Erl.

CONTEÚDO

  • O que é SOA?
  • Características fundamentais
  • Tipos de SOA
  • Influência histórica
  • Benefícios estratégicos
  • Impactos comuns
  • Serviços
    • Encapsulamento, Relacionamento, Comunicação e projeto
    • Princípios de Orientação a serviços
  • Web Services e SOA primitivo
  • Web Services e SOA Contemporâneo
  • Formas de Implementação

DOWNLOAD

  1. Apresentação.
  2. Complementos – não disponível

MVC Design Pattern with DOS Batch Files!

I’ve been speechless for some moment just before this blog comes up. In the unlimited world of programming languages, I’ve met DoD (DOS on Dope). It despises Ruby on Rails, .NET, PHP, or any of those fancy web frameworks and give you an web site with MVC design pattern just using DOS. Amazing work!

WCF error: the underlying connection was closed an unexpected error occurred on a receive

I don’t like to post solution as comment industry, strategies, patterns… But this solution gave me happiness. I want to share it because these error has so many reasons to happen I haven’t found solution anywhere.

I had the folowing problem:

System.Exception: An error occurred while receiving the HTTP response to http://soaAdress.com. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details..

which datails:

the underlying connection was closed an unexpected error occurred on a receive

I thought my client application was fine because it closes the connection. Last time I had problem with WCF calls, the calls stopped at 11º one because we can’t keep a stabilished connection to perform more than 10 calls. But this time I couldn’t close. The service contract has 4 operations:

[OperationContract(IsOneWay = false)]
[TransactionFlow(TransactionFlowOption.NotAllowed)]
myEntity GetComission(long id);

[OperationContract(IsOneWay = false)]
[TransactionFlow(TransactionFlowOption.NotAllowed)]
myEntity GetComissionPartial(long id);

[OperationContract(IsOneWay = false)]
[TransactionFlow(TransactionFlowOption.NotAllowed)]
myEntity GetComissionAuto(long id);

[OperationContract(IsOneWay = true)]
void CloseOderProcess(Contracts.CloseMessageOrder input);

The dump file was terrible to read, but I found something:

“The message could not be processed. This is most likely because the action
http://schemas.xmlsoap.org/ws/2005/02/trust/RST/SCT/Cancel’ is incorrect or
because the message contains an invalid or expired security context token or
because there is a mismatch between bindings. The security context token
would be invalid
if the service aborted the channel due to inactivity. To
prevent the service from aborting idle sessions prematurely increase the
Receive timeout on the service endpoint’s binding.”

It let me to a conclusion, if the service operation is IsOneWay = true, I shouldn’t close it. The service wouldn’t have the security token anymore as there is a very short connection between service and consumer and the message is operated asynchronously as stated in the contract. So I removed the .close operation and things are working now.

Surt Framework

Has been three years since I started studies on advanced way to start software projects. This time gave me some authority on framework design using modern methodologies and simples technology that works!

I’ve been usin this research and some of the past projects I’ve done and now it became mature and I hope I can share it. Take a look at this page. It talks about what it is, technologies and some of methodology I use. I hope to increment details as the time goes on.

I’m so happy because I had migrated one poor enterprise software to a highly structure one, by changing usability, patterns, layers in few weeks working alone. Badly saying, migrating some 20 files (lists, froms, data screens) with lots of complexity to 50 files with simplicity, good usability, navigation, ajax, SOA Services. Users were amazed and they don’t know I did much more, with less people, no error, and the system works best.

The dilemma of the agnostic software architect

We have been spreading a different thinking in our customers advocating that their products needs to be somehow technology agnostic. So what that could possible mean?

A meeting with my customer put forward an important discussion about whether they should build or buy off-the-shelf components to integrate into the new product release which radically change its environment from to web. I mean from Delphi to ASP.NET.

They were conducting their migration process based only in the ASP.NET available functionalities. They’ve shown a research, comparing a past Delphi solution with a possible .net solution. They pointed out a range of off-the-shelf components like ajaxControlToolkit and many others which they wanted to purchase.

Desktop and web solution are different paradigms indeed and demands different kinds solution and methodologies. They didn’t see that all they need would be simply being done at home.

One may think about limitation of the web, and we believe about the wide range solution the web can provide. Take the example of some industry leaders like Amazon.com and Google. Do you really believe they follow Microsoft & Java framework versions and keep using off-the-shelf components extensively? Actually YOU may use what they provide.

I’m not saying that we shouldn’t use off-the-shelf components, there’s nothing wrong with that. We need to balance about its usage. For example, the Enterprise Library Logging Application Block offer many options to log information into the text files, event viewer, e-mail and databases. But my application only needs to write log in the event viewer. Having many options doesn’t limit my application IF IN THE FUTURE I want to change log directions. Future seems so distance, so intangible that it must be preferable write a small components with fewer lines providing same functionality instead of having a monster component with lots functionalities and overhead. (Have you had time to look their code? Probably not but you may want if your product start to perform badly or if you change from one version to another, or if you may want to use MySql instead of the SqlServer!)

At the end, what we always are looking foward is a clean path to quickly deliver value and avoid spending time working bugs out.

Contract-First using a custom DSL

Has been while since my last post.  There’s nothing easy about doing consulting and keep researching. The fact is, I was misplaced on contract first approach and I know I’ll put forward some conclusions.

  1. Contract-first approach demands strong organization and standardization.
  2. You can’t count on off-the-shelf tools.  You likely develop your own Domain Specific language.
  3. I left off-the-shelf code generation tool (I’ll not post it’s name in this case) for Industry Standards. Last approach was about taking tables structure and give life to application by generating all necessary files.  I was too coupled to the database platform. Different database versions & vendors, operation systems used to gave a lot of headaches. By Tomas Erl literature I was doing a negatively case of coupling. I started to live all the promisses bad dreams by keep going on this approach. By the next image you can conclude that my code generation tool was really coupling to many different vendors technology, from the source (tables) to the engine (code generation framework) to the assets (text templates).

    DSL Framework Vision

  4. I Start work with XML Schema Industry standard. It was need good ideas to be organized and produce high quality code. It increase the need for methodology and standardization.
    The economies of reuse was increased because it became a flexible structure.

    Code Generation Framework Vision based on Industry Standards

    Code Generation Framework Vision based on Industry Standards

Change to Industry Standards was a good thing because it is wisdom and experienced approach recommendation  from serious institutions which researches are solid results that doesn’t keep changing every year as when we are based on technology platform which we start to accompany their bugs, different versions and releases of trial and error.

Achieving high productivity inside environments is important to understand and improve process. Build a Dialect language or Domain Specific Language based on industry standards instead on technology platform will easy lot of implementation pain as meaning of control but also will demand most talent engineering to help out on conceptions. It is not a easy path but has no better ROI.

References

  • Software Product Lines: Practices and Patterns – Chapter 1,4 – Paul Clements and Linda Northrop/2001
  • Software Engineering (7th Edition) – Chapter 18 – Ian Sommerville/2004
  • Software Factories: Assembling Applications with Patterns, Models, Frameworks, and Tools – Chapter 2,7,10,11,14,15 – by Jack Greenfield, Keith Short, Steve Cook, and Stuart Kent (Paperback – Aug. 16, 2004)
  • Service-Oriented Architecture (SOA): Concepts, Technology, and Design – Chapter 10,11,14,15 – Thomas Erl/2005
  • SOA Principles of Service Design – Chapter 6,7 – Thomas Erl/ 2007
  • Domain Specifc Languages – Martin Fowler -http://www.martinfowler.com/bliki/DomainSpecificLanguage.html (at 06/2010)
Follow

Get every new post delivered to your Inbox.